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Foreword 


In this volume we present the major, policy-oriented report of OTA’s assessment, 
The Implications of Cost-Effectiveness Analysis of Medical Technology. Begun in Oc- 
tober 1978, the assessment analyzes the feasibility, implications, and usefulness of 
cost-effectiveness and cost-benefit analysis in health care decisionmaking. 


In addition to this main report, there will be five background papers: 1) one cov- 
ering methodology and literature issues, 2) a psychotherapy case study, 3) a diagnostic 
X-ray case study, 4) 17 other case studies of individual medical technologies, and 5) a 
review of international experience in managing medical technology. Another related 
study was published in September of 1979: A Review of Selected Federal Vaccine and 
Immunization Policies. 
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Technology, was prepared by OTA staff. In preparing this report, OTA consulted 
with members of the advisory panel for the assessment, with the authors of the case 
studies prepared for the assessment, and with numerous other experts in economics, 
medicine, ethics, and health policy. 
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1: 
Summary and Policy Options 


Nature never gives anything to anyone; everything is sold. It 
is only in the abstractions of ideals that choice comes without 
consequences. 


Ralph Waldo Emerson 
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Summary and Policy Options 


The rapid and continuing growth of expendi- 
tures is a central issue in many policy decisions 
concerning the medical care system of the 
United States. Policymakers, health profession- 
als, and consumers are seeking ways to control 
this growth while simultaneously improving the 
quality of health care. Increasingly, the use of 
cost-effectiveness analysis/cost-benefit analysis 
(CEA/CBA) is being advocated as a possible 
means of making the medical care system more 
efficient. In particular, this technique is sug- 
gested for use in health care programs—for ex- 
ample, by the medicare program in its reim- 
bursement coverage decisions. Nevertheless, a 
great deal of confusion and disagreement sur- 
rounds the implications and feasibility of apply- 
ing CEA/CBA in health care. To aid in their de- 
cisions concerning the possible use of CEA/ 
CBA in Federal health programs, the Senate 
Committees on Labor and Human Resources 
and on Finance asked OTA to explore the appli- 
cability of CEA/CBA to medical technology. 


In the assessment, three major issues are ex- 
amined: 1) the general value of CEA/CBA in de- 
cisionmaking about the use of medical technol- 
ogy; 2) the methodological strengths and short- 
comings of the technique; and 3) the potential 
for initiating or expanding the use of CEA/ 
CBA in six health care programs—reimburse- 
ment coverage, the Professional Standards Re- 
view Organizations (PSROs), health planning, 
market approval for drugs and medical devices, 


EVALUATING COSTS AND BENEFITS 


All decisions have consequences. Usually, 
however, in most decisionmaking processes 
only a fraction of all potential consequences is 
taken into account. The inherent complexities 
and uncertainties associated with many deci- 
sions make it extremely difficult to identify and 
weigh all possible consequences. In general, 
however, the quality or validity of decisions can 


R&D activities, and health maintenance organi- 
zations (HMOs)—and, most importantly, the 
implications of any expanded use. 


The prime focus of the assessment is on the 
application of CEA/CBA to medical technol- 
ogy, i.e., the drugs, devices, medical and surgi- 
cal procedures used in medical care, and the or- 
ganizational and support systems within which 
such care is provided. Except in a background 
paper on psychotherapy, the report does not ad- 
dress psychosocial medicine. Other factors in- 
fluencing health, such as the environment, are 
not directly covered either. The findings of this 
assessment, though, might very well apply to 
health care resource decisionmaking in general, 
and with modification, to other policy areas 
such as education, the environment, and occu- 
pational safety and health. 


This OTA assessment finds that CEA/CBA 
cannot serve as the sole or primary determinant 
of a health care decision. Decisionmaking could 
be improved, however, by the process of identi- 
fying and considering all the relevant costs and 
benefits of a decision. At present, using the ap- 
proach or process of CEA/CBA in decisionmak- 
ing may be more helpful than the rigid and for- 
mal application of CEA/CBA study results in 
health care program decisions. It is unrealistic, 
moreover, to expect that CEA/CBA, in itself, 
would be an effective tool for reducing or con- 
trolling overall expenditures for medical care. 


be increased by analysis that forces a structuring 
of the decision process—that provides a consist- 
ent framework for identifying and considering 
as many of the relevant costs and benefits as is 
feasible. 


The public, or governmental, sector is called 
upon to make certain decisions that are imprac- 
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tical for the private sector to make. Examples of 
these are decisions concerning national defense 
or air pollution control, neither of which is 
amenable to being traded in the marketplace. 
Other decisions are made by the public sector 
for social reasons such as assuring equitable dis- 
tribution of what may be considered essential 
goods (e.g., health care for the elderly). Because 
conventional private sector techniques, such as 
capital budgeting and_ return-on-investment 
analysis, are insufficient for these decisions, 
techniques such as CBA and CEA have been de- 
veloped. In the medical care area, CEA/CBA is 
designed to integrate the economic aspects of a 
decision with the health aspects of that decision. 
Consequently, it should not be considered sim- 
ply an economic tool. 


There are two basic types of health care re- 
source allocation decisions which in theory 
could benefit from a CEA/CBA process. The 
first are decisions made within a fixed or pro- 
spectively set budget, such as those made by 
HMOs. The second are decisions made in the 


absence of a direct budget constraint, such as 
those made for medicare reimbursement or in 
health planning. 


In the former—allocation decisions made 
within a budget—tradeoffs must be made, since 
not all projects can be funded. The projects that 
promise to deliver more benefits for the cost 
should be more attractive than those projects 
expected to deliver fewer benefits. In these deci- 
sions, the function of CEA/CBA would be to il- 
luminate the decision process and to require that 
implicit judgments be made more explicitly, 
thus forcing external examination of the as- 
sumptions and values placed on decision vari- 
ables. 


In nonconstrained decisions, direct tradeoffs 
between competing projects often are unneces- 
sary. Consequently, a function of CEA/CBA in 
these decisions would be to force consideration 
of economic factors. In health planning deci- 
sions, for example, planners would be asked to 
consider not only whether a service is needed 
but also whether it is worth the cost. 


COST-BENEFIT ANALYSIS AND COST-EFFECTIVENESS ANALYSIS 


The terms CBA and CEA refer to formal ana- 
lytical techniques for comparing the positive 
and negative consequences of alternative ways 
to allocate resources. In practice, the compar- 
ison of costs and benefits is accomplished 
through various analytical approaches, which 
comprise a spectrum ranging from sophisticated 
computer-based analysis using large amounts of 
epidemiological and other data to partially in- 
tuitive, best-guess estimates of costs and bene- 
fits. Some analyses may take into account the 
results of clinical trials of a technology and 
model the technology’s effect on health out- 
comes. Others may assume that the alternative 
technologies under study have equal effective- 
ness and concentrate on the difference in costs 
involved. 


Thus, there is a continuum of analyses that 
examine costs and benefits. At one end of the 
continuum are what will be referred to as “net 
cost” studies. In these studies the emphasis is on 


costs, and net cost studies in the past have often 
assumed benefits or efficacy to be equal. At the 
other end of the continuum are analyses that at- 
tempt to relate the use of the technologies under 
study to specific health-related outcomes and to 
compare the costs of the technologies to the dif- 
ferences in health benefits. CBA and CEA com- 
prise the entire set of analytical techniques— 
differentiated by the specifics of what costs and 
benefits are considered and how they are ana- 
lyzed—on this continuum. 


The principal distinctions between CEA and 
CBA lie in the valuation of the desirable conse- 
quences of a decision, in the implications of the 
different methods of that valuation, and usually 
in the scope of the analysis. In CBA all costs and 
all benefits are valued in monetary terms. Thus, 
conceptually, CBA can be used to evaluate the 
“worth” of a project and would allow compar- 
ison of projects of different types (such as dams 
and hospitals). In CEA, the health-related ef- 


fects of programs or technologies are not valued 
in monetary terms but rather are measured in 
some other unit (such as years of life gained). A 
CEA, therefore, does not result in a net mone- 
tary value for a project. Instead, it produces a 
measure of the cost involved in attaining some 
desirable health-related effect. Conceptually, 


FINDINGS 


General 


Most of the specific findings presented below 
relate to the two major, general findings of the 
OTA assessment: 


e Performing an analysis of costs and bene- 
fits can be very helpful to decisionmakers 
because the process of analysis gives struc- 
ture to the problem, allows an open consid- 
eration of all relevant effects of a decision, 
and forces the explicit treatment of key 
assumptions. 

e CEA/CBA exhibits too many methodologi- 
cal and other limitations, however, to justi- 
fy relying solely on the results of formal 
CEA/CBA studies in making a decision. 
Thus, although CEA/CBA is useful for as- 
sisting in many decisions, it should not be 
the sole or prime determinant of a decision. 


CEA/CBA is viewed by different parties as 
ranging in usefulness from obfuscating the perti- 
nent issues in a decision process at one extreme 
to illuminating and synthesizing these issues so 
well that the technique can be used to make de- 
cisions at the other extreme. There is, however, 
a middle position which maintains that the tech- 
nique could be helpful in structuring informa- 
tion and that this information should be only 
one of several components of a decision process. 
The OTA case study on the artificial heart, for 
example, lays out many of the factors to be con- 
sidered in decisions regarding continued funding 
of R&D of this technology. But those decisions 
will also be dependent on a number of other po- 
litical and social factors that are difficult to ana- 
lyze systematically. 


Both extreme positions mentioned above per- 
tain to the use of CEA/CBA as a formal, struc- 
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CEA permits direct comparison of only those 
programs or technologies that share similar ob- 
jectives. This OTA assessment uses the term 
CEA/CBA to refer to both of these techniques 
because the findings below apply generally to 
both. 


tured analysis which is oriented toward a 
bottom-line answer, such as a cost-benefit ratio. 
Such a bottom line, however, often hides many 
important value judgments, thus providing a 
seemingly unambiguous answer that may rest 
on ambiguous data or assumptions. 


Advocates of the middle position propose 
that CEA/CBA be used within the context of ac- 
cepted principles of analysis in order to illum- 
inate the costs and the benefits of a decision but 
not necessarily to aggregate and weigh them. 


The findings of this and any other assess- 
ments of the past usefulness or current potential 
of CEA/CBA in health care decisionmaking 
should be kept in perspective: Because there has 
been little experience with the use of CEA/CBA, 
these findings are based on very little definite in- 
formation. Therefore, any of the findings or 
projections of the usefulness or implications of 
CEA/CBA in this report could be proven wrong 
as more experience accumulates. In fact, one of 
the priorities for future examinations of the role 
of CEA/CBA in health care should perhaps be 
small, controlled experiments, or demonstra- 
tions, of its potential use (see Option 10 below). 


Methodology 


There is no set combination of specific analyt- 
ical elements that form a standard CEA or CBA 
methodology. A standard or rigid methodolo- 
gy, however, is to be neither expected nor de- 
sired. OTA found a wide variation in the forms 
of CEA/CBA analyses actually conducted to 
date. Most of the analyses reviewed seem to be 
academic exercises, infrequently connected to 
the policy process. In addition, the literature on 
CEA/CBA indicates that a great many of the 
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analyses conducted tend to be on the “net cost” 
end of the CEA/CBA continuum, that is, anal- 
yses in which health outcomes were not taken 
into explicit account. In part, this may be a re- 
flection of the paucity of data on the efficacy, 
safety, and appropriate use of medical tech- 
nology. 


OTA’s assessment does find, however, gen- 
eral agreement on a set of 10 principles of 
analysis that could be used to guide the conduct, 
evaluation, or use of CEA/CBA studies. OTA 
believes that use of generally accepted principles 
is important, largely because of the basic meth- 
odological limitations of CEA/CBA. For exam- 
ple, although the results of a CEA/CBA are of- 
ten dependent on the discount rate chosen, there 
is no general agreement on what discount rate 
should be used under what circumstances. Fur- 
thermore, as primarily an efficiency-oriented 
technique, CEA/CBA is weak in the areas of 
equity and other ethical considerations. Finally, 
CEA/CBA must often address issues of great 
uncertainty, such as disease progression, patient 
compliance rates, differing responses to tech- 
nology by differing population groups, and so 
on. These and other limitations must be kept in 
mind and must to the maximum extent feasible 
be compensated for by techniques such as sen- 
sitivity analysis (see #8 below). The 10 basic 
principles of analysis are: 


1. Define Problem.—The problem should 
be clearly and explicitly defined and the 
relationship to health outcome or status 
should be stated. 

2. State Objectives.—The objectives of the 
technology being assessed should be ex- 
plicitly stated, and the analysis should 
address the degree to which the objec- 
tives are (expected to be) met. 

3. Identify Alternatives.—Alternative 
means (technologies) to accomplish the 
objectives should be identified and sub- 
jected to analysis. When slightly dif- 
ferent outcomes are involved, the effect 
this difference will have on the analysis 
should be examined. 

4. Analyze Benefits/Effects.—All foresee- 
able benefits/effects (positive and nega- 
tive outcomes) should be identified, and 


when possible, should be measured. 
Also, when possible, and if agreement 
can be reached, it may be helpful to val- 
ue all benefits in common terms in order 
to make comparisons easier. 

5. Analyze Costs.—All expected costs 
should be identified, and when possible, 
should be measured and valued in dol- 
lars. 

6. Differentiate Perspective of Analysis.— 
When private or program benefits and 
costs differ from social benefits and costs 
(and if a private or program perspective 
is appropriate for the analysis), the dif- 
ferences should be identified. 

7. Perform Discounting.—All future costs 
and benefits should be discounted to 
their present value. 

8. Analyze Uncertainties.—Sensitivity 
analysis should be conducted. Key vari- 
ables should be analyzed to determine 
the importance of their uncertainty to the 
results of the analysis. A range of possi- 
ble values for each variable should be ex- 
amined for effects on results. 

9. Address Ethical Issues.—Ethical issues 
should be identified, discussed, and 
placed in appropriate perspective relative 
to the rest of the analysis and the objec- 
tives of the technology. 

10. Discuss Results.—The results of the anal- 
ysis should be discussed in terms of va- 
lidity, sensitivity to changes in assump- 
tions, and implications for policy or deci- 
sionmaking. 


In addition to conforming to these 10 prin- 
ciples, all quantitative analyses should speci- 
fy data sources and be written as clearly and 
nontechnically as possible. They should also be 
subjected to review—including public scruti- 
ny when appropriate—especially regarding as- 
sumptions upon which their outcomes may rest. 


OTA found that many of the methodological 
limitations of CEA/CBA are often hidden by 
the practice of deriving a cost-benefit or cost- 
effectiveness ratio—that is, a numerical bottom- 
line. Therefore, the possibility of not aggregat- 
ing the often complex sets of calculations should 
be investigated. Instead of aggregating, analyses 


might be done by explicitly listing or ARRAY- 
ING all the elements which are included in or 
would be affected by a decision. Where costs 
and effects could be quantified, that would be 
done; where they could be combined, that 
would be done. But no effort would be made to 
arrive at a single combined value when one or 
more important nonquantifiable variables that 
would have to be left out or relegated to a foot- 
note. Such an array method of analysis would 
mean that decisionmakers would have a greater 
number of elements to consider, but it would 
also mean that intangible or nonquantifiable 
factors would be more explicit and, thus, more 
likely to be taken into consideration. 


The findings of this assessment, especially 
methodological ones, focus primarily on the 
practical implications of CEA/CBA for health 
policy. The report is not written for the aca- 
demic research community. The general prin- 
ciples above apply to analysis for policy use. In 
no way, however, should the findings be taken 
to mean that complex, sophisticated analysis is 
always unnecessary or superfluous. Advance- 
ment of the state-of-the-art of CEA/CBA re- 
quires ongoing and sophisticated research. Cur- 
rent research on the development and validation 
of sets of indexes of the health status of a pop- 
ulation, for example, appears very valuable. 
Use of such complex aspects of CEA/CBA, 
however, may require greater technical capa- 
bilities than most health programs currently 
possess. Addressing a more limited approach to 
analysis, then, seems appropriate for the goal of 
this assessment. But it does not diminish the 
need for more complex approaches in research 
or other specialized circumstances. 


Applying CEA/CBA in Health Care 


As stated above, CEA/CBA can be more val- 
uable to health care decisionmaking when it 
serves as a problem structuring process than 
when it becomes the primary factor, with nu- 
merical results, of a decision. Furthermore, 
CEA/CBA potentially can be more valuable for 
decisionmaking under a constrained budget, 
when tradeoffs have to be made directly, than 
when constraints are nonexistent or very indi- 
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rect. In neither case, however, would CEA/ 
CBA necessarily function as an effective cost- 
constraining mechanism or tool. Under the 
budget system, the budget itself would be the 
constraining mechanism. Under the noncon- 
strained system, since no direct tradeoffs are re- 
quired, no direct limit on expenditures is set. 
CEA/CBA might, however, change the mix of 
expenditures. Especially under a budget system, 
technologies might be substituted for one 
another as a result of analysis. 


The context within which a decision will take 
place must be specified before any judgment of 
the usefulness or applicability of CEA/CBA can 
be made. For example, does the decision relate 
to a technology at an early stage in its lifecycle 
such as bone marrow transplants? Or does it 
concern an established technology, such as cer- 
vical cancer screening? Is the technology in 
question a diagnostic technology, such as the 
CT scanner, or a therapeutic one, such as renal 
dialysis? The possibility of affecting the course 
of a technology’s diffusion and use might be 
greater in early stages of its development, but 
the uncertainties about its health effects and its 
costs will generally be greater. Thus, it may be 
possible to do a more valid or certain CEA/ 
CBA later in the technology’s lifecycle, but the 
information gained may be less valuable for 
public policy. The tradeoff required will vary 
depending on the specifics of the technology and 
the policy decision to be made. Similarly, diag- 
nostic technologies are often more difficult to 
study than other types because of the uncertain- 
ties involved in linking their use to health out- 
comes. Thus, studies of diagnostic technologies 
often tend toward the “net cost” end of the 
CEA/CBA spectrum, where the measures of 
outcome or benefit may be numbers of tests per- 
formed or levels of diagnostic accuracy. 


Applying CEA/CBA in Specific Health 
Care Programs 


OTA examined the current and potential use 
of CEA/CBA or related techniques in six health 
care programs (see chs. 5 through 10). Although 
informal and often implicit analysis of costs and 
benefits frequently takes place in health policy 
formulation, OTA found very little formal use 
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of CEA/CBA in the programs studied. In sever- 
al of the areas, cost itself has played little or no 
role in policy decisions. 


Reimbursement programs such as medicare 
and Blue Cross/Blue Shield, when deciding 
what technologies will be covered, concentrate 
on criteria that generally do not explicitly in- 
clude costs (e.g., stage of development of the 
technology and acceptance by the medical com- 
munity). Under medicare, initial responsibility 
for identifying questions about whether a tech- 
nology should be covered lies with a system of 
local contractors who administer the program. 
When not resolved at the local level, a question 
is referred to the Health Care Financing Admin- 
istration (HCFA), which may seek a recommen- 
dation from the Public Health Service (PHS). 
PHS has traditionally used four criteria in its 
recommendations: efficacy, safety, stage of de- 
velopment, and acceptance by the medical com- 
munity. Other health insurance programs, such 
as Blue Cross/Blue Shield, operate similarly. 


The possibility of expanding coverage criteria 
to explicitly include costs or cost effectiveness is 
being examined by HCFA and PHS. The first 
question to be answered is whether there is a 
legal basis for any such inclusion. Currently, the 
language of the Social Security Act requires the 
medicare program to pay for services that are 
“reasonable and necessary.” There is no defini- 
tive interpretation of whether that language 
means that the relative “cost effectiveness” of a 
particular technology might make it unreason- 
able or unnecessary. 


Most current reimbursement programs are ex- 
amples of programs without direct budget con- 
straints. Each reimbursement coverage decision 
does not involve a tradeoff. Approval of one 
technology does not mean that another will not 
be covered. In a very real sense, the existing 
reimbursement system is an open-ended system 
of financing medical care. 


The PSRO program was enacted to assure 
that health services provided under medicare, 
medicaid, and certain other programs are medi- 
cally necessary, meet professionally recognized 
standards of care, and are provided at the most 
economical level consistent with quality care. 


CEA/CBA approaches have theoretical applica- 
bility in three areas: 1) the development of 
standards of care against which actual practices 
are judged, 2) the internal management of in- 
dividual PSROs, and 3) evaluations of the na- 
tional PSRO effort. Cost-effectiveness criteria 
have not been directly incorporated into stand- . 
ards of care except in a few instances (e.g., res- 
piratory therapy), but it appears possible for 
PSROs to do so, although the studies would 
most likely have to be conducted elsewhere. 
PSROs generally do not have the analytical 
capabilities for CEA/CBA. Net cost techniques 
have been used to evaluate whether the savings 
achieved outweigh the costs of administering 
the review activities of the overall PSRO pro- 
gram. These analyses, which often reach con- 
tradictory conclusions, do not, however, ex- 
amine costs in relation to changes in health out- 
comes that may result from PSRO reviews. Nor 
do they address the fact that even the most op- 
timistic reports of savings represent an infinites- 
imal portion of total medicare and medicaid ex- 
penditures. 


Because individual PSROs operate under a 
program management budget, incentives ex- 
ist for them to use cost-effectiveness-like ap- 
proaches in choosing areas in which to concen- 
trate their review activities. 


In contrast to PSROs, in the area of health 
planning, the National Health Planning and 
Resources Development Act, with its amend- 
ments, explicitly states that resources are to be 
allocated in a more efficient manner and that 
health planners should weigh both costs and 
benefits in their decision processes. The Health 
Resources Administration (HRA) is emphasiz- 
ing a more analytical approach to health plan- 
ning, especially in regard to capital budgeting. 
OTA found that State health planning and de- 
velopment agencies (SHPDAs) and health sys- 
tems agencies (HSAs) are for the most part still 
primarily health “needs” oriented. An OTA sur- 
vey of planning agencies indicated that few 
agencies, even those in the vanguard of using 
economic analysis for allocation of resources, 
are going beyond the practice of considering 
only capital costs. There is a slight trend, how- 
ever, for the most analytically advanced staffs 


to consider the marginal (or incremental) costs 
associated with changes in use of a technology. 
The analysis that took place around the CT 
scanner is a good example of that. OTA discov- 
ered no HSAs that explicitly balance costs with 
health benefits in, for example, certificate-of- 
need recommendations. Thus, although there 
appear to be no legal barriers to its use, CEA/ 
CBA has not been much applied. In health plan- 
ning, as in reimbursement, there is no direct 
budget constraint, i.e., the area served by an 
HSA is not operating with a fixed or predeter- 
mined amount of resources to be spent on health 
care. Few pressures, therefore, act on planning 
agencies to force consideration of how to get the 
most health effect for the fewest dollars. 


Market approval for drugs and medical de- 
vices, by the Food and Drug Administration 
(FDA), is an area where Congress has specified 
the decision criteria, which explicitly do not in- 
clude costs. FDA is required to regulate the mar- 
ket introduction of drugs and devices on the 
basis of effectiveness (efficacy) and safety. FDA 
has not formally used cost-effectiveness or any 
other economic criteria to evaluate drug or 
device applications. The primary purpose of 
FDA is to protect the public from unsafe and in- 
efficacious products. Although the Agency’s 
processes do have indirect influence on the way 
resources are allocated, there are several factors 
arguing against the incorporation of explicit 
economic criteria in its decision processes. Per- 
haps the most important of these is that by in- 
corporating such criteria, FDA might be extend- 
ing its regulatory functions beyond those envi- 
sioned by Congress. Further, the administrative 
burden and demands of doing so would be enor- 
mous. 


The Federal health R&D effort encompasses a 
broad spectrum of activities and involves sever- 
al Federal agencies. At one end of the spectrum 
is biomedical and behavioral research which is 
supported by the National Institutes of Health, 
and at the other end is health services research 
such as that supported by the National Center 
for Health Services Research, the National Cen- 
ter for Health Statistics, and HFCA. Lying 
somewhere in the middle of the spectrum and in- 
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corporating both end points is research sup- 
ported by the National Center for Health Care 
Technology. The Federal agencies involved in 
health R&D rarely use explicit CEA/CBA con- 
siderations to set research priorities, to allocate 
research resources, or to evaluate the results of 
research. The uncertain end products of much 
research makes it difficult to conduct CEA/ 
CBA. Technologies being assessed or to be de- 
veloped are often at an early stage of the life- 
cycle. A CEA/CBA-like approach, with no re- 
quirement for aggregation of variables, may be 
more applicable for R&D than is rigid, tradi- 
tional CEA/CBA. The more basic the level of 
research, the less applicable the techniques be- 
come, owing to the increased uncertainties. 
Ironically, it may frequently be more desirable 
to assess a technology, including its cost impli- 
cations, earlier in its development rather than 
later. 


HMOs are both insurers and providers of 
medical care. They are an example of a program 
operating under a constrained budget and have 
a direct economic incentive to provide ‘‘cost-ef- 
fective” care. However, a preliminary OTA ex- 
amination of HMO’s decisionmaking criteria in- 
dicates that, at least in part because HMOs op- 
erate in a predominantly fee-for-service envi- 
ronment and must compete for enrollees, these 
organizations do not commonly weigh health 
benefit against cost in deciding what medical 
services to offer. The actual analytical approach 
used by HMOs seems to be related to “net cost” 
techniques. Although HMOs do not always 
consider health benefits, their use of a “net cost” 
approach suggests that they view the provision 
of technology in terms of efficiency. They seek 
ways to provide benefits comparable to fee-for- 
service medicine at the lowest cost feasible. The 
current potential for use of CEA/CBA by 
HMOs, weighing health benefits against costs, 
does not appear to be as large as the existence of 
direct budget constraints would predict. That 
situation may change, however, as more experi- 
ence is gained with CEA/CBA and as HMOs in- 
creasingly encounter competition with each 
other in addition to fee-for-service health care 
providers. 
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POLICY OPTIONS 


Options for congressional consideration fall 
into two categories: 1) those that relate to the 
current possibilities for using CEA/CBA in pol- 
icy formulation and decisionmaking (Options 1 
through 6), and 2) those that relate to the devel- 
opment of CEA/CBA techiques in themselves 
(Options 7 through 10). 


Options Relating to Current 
Possibilities for Use of CEA/CBA 


Options 1 through 6 follow a four-stage pro- 
gression of confidence in the current applicabili- 
ty of CEA/CBA to health care programs—from 
a perception that its use should be discouraged 
(Option 1), to a neutral, status quo attitude 
(Option 2), to cautious encouragement (Option 
3), to a positive belief that CEA/CBA can be ef- 
fectively applied (Options 4 through 6). Options 
1 through 3 present general approaches that are 
mutually exclusive for any given program area. 
Each of Options 4 through 6, however, relates 
to a specific program area. 


OPTION 1 
Discourage or prohibit the explicit inclusion of 
cost-effectiveness criteria in specified health care 
resource allocation decisions. 


The implicit weighing of the costs and ben- 
efits of resource allocation decisions cannot— 
and should not—be legislated out of the policy 
process. It is inherently a part of that process. 
What this option does, therefore, is signal that 
the techniques available to formalize the now 
informal process of weighing these costs and 
benefits are not regarded as sufficiently valid to 
justify their use. 


For those types of decisions where costs have 
not previously been explicitly considered to any 
significant extent (such as medicare coverage of 
specific technologies), this option would reflect 
a positive decision not to include costs explicit- 
ly. For decisions made within a constrained 
budget, where the costs of alternative alloca- 
tions of resources are automatically, though 
often implicitly, factored in, this option would 
be a statement that existing methods of consid- 
ering costs are either adequate or would not be 


improved upon by mandatory use of formal 
analytical techniques. 


Reflected in this option are a recognition of 
both the inherent methodological limitations of 
CEA/CBA and the weaknesses that result from 
the fact that CEA/CBA is still an evolving meth- 
odology; a belief that other factors in the deci- 
sionmaking context are more important than 
CEA/CBA-derived information; and a desire to 
avoid the possible misuse of CEA/CBA. These 
considerations, along with considerations of the 
country’s limited experience with making health 
care allocation decisions based on CEA/CBA, 
and the expense associated with conducting 
large numbers of CEA/CBAs required for deci- 
sionmaking, all support this option. 


However, if the health care system is in fact 
operating in an era of limitations on resources 
and if the concern over rising health care costs 
continues to lead to pressures for better balanc- 
ing of costs with outcomes in the use of medical 
technology, this option becomes much less at- 
tractive. It might eliminate most of the current 
and future developmental work on techniques 
for comparing costs to benefits. It would force 
reliance on current methods of making deci- 
sions. If Congress is not satisfied with the 
allocation and use of medical technology and 
other health care resources and not satisfied 
with the cost implications of such allocations, 
some method for more explicitly forcing a 
weighing of costs with benefits may need to be 
developed. (Options 7 through 10 below could 
be a step in that direction.) 


Thus, this option should not be adopted if 
Congress believes that more explicit balancing 
of costs and benefits is necessary and that CEA/ 
CBA or similar techniques hold the potential to 
contribute to that process. Rejection of this op- 
tion does not mean that immediate mandatory 
use of CEA/CBA should take place; it simply 
does not shut the door to that possibility or to 
the possibility of encouraging methodological 
development, experiments with increased use of 
CEA/ CBA, and increased use of formal analyt- 
ical techniques for balancing costs with benefits. 


OPTION 2 
Allow current trends in development and use of 
CEA/CBA to continue, but make no changes in 


legislative requirements or prohibitions relating 
to the use of CEA/CBA. 


This is basically a status quo option, but it 
recognizes that the field of CEA/CBA is under- 
going methodological change and that analysts 
and policymakers are paying increasing atten- 
tion to the potential uses of the techniques. 
These trends would continue under this option. 
(They could be encouraged through implemen- 
tation of any or all of Options 7 through 10.) 
But no changes in legislative language regarding 
the formal, explicit use of CEA/CBA in pro- 
gram decisions would be made. Since OTA 
found that very little formal use of CEA/CBA 
currently takes place, that nonuse would most 
likely continue into the near future. 


Under this option, health care programs 
would be allowed to examine the possibility of 
using more explicit costs and benefits informa- 
tion. But they would not be required or encour- 
aged to do so by Congress. 


In medicare reimbursement decisions, this op- 
tion would enable HCFA and PHS to continue 
their investigation into the possibility of adding 
cost effectiveness to the list of coverage criteria. 
At the same time, the option recognizes that the 
negative aspects of adding cost effectiveness as a 
criterion (discussed below under Option 4) are 
quite serious. It would allow demonstration 
projects (as presented in Option 10) to take 
place before a general decision to change current 
reimbursement policies is made. 


Current legislative language for market ap- 
proval of drugs and medical devices would con- 
tinue to specify only safety and effectiveness as 
primary decision criteria. This situation seems 
to reflect congressional intent to date and would 
avoid the problems associated with changing 
the laws to include cost effectiveness (as exam- 
ined in Option 6 below.) 


Similarly, other health care programs would 
be allowed to continue investigating the possi- 
bility of using more explicit costs and benefits 
information. But they would not be required or 
encouraged to do so by Congress. For example, 
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the health planning program could continue to 
provide technical assistance regarding cost and 
cost-effectiveness analysis to local HSAs. Also, 
Congress could encourage or discourage the es- 
tablishment and expansion of HMOs without 
mandating or prohibiting their use of cost-effec- 
tiveness criteria for decisionmaking regarding 
technology. 


Selection of this option would in part reflect 
the view that the pros and cons of the other op- 
tions are too uncertain to permit a definite deci- 
sion on prohibition or on active encouragement 
of use of CEA/CBA. But it also would reflect a 
belief that some method of more explicit con- 
sideration of costs and benefits is needed. 


OPTION 3 
Encourage the explicit consideration of costs 
and benefits in resource allocation, but do not 
require it as a formal decision criterion. 


Selection of this option requires a more favor- 
able view of the current potential of formal 
analysis than does the previous option. This op- 
tion, in effect, says that health care is at the 
point where costs should be explicitly compared 
to benefits when decisions about resource allo- 
cation are made, and that the techniques that 
are available to make such comparisons are use- 
ful—though not well enough developed to be 
used as mandatory aspects of the decisions. This 
option goes beyond the status quo. Congress, 
by adopting this approach, would be explicitly 
approving the approach of CEA/CBA. Decision 
areas such as health planning would receive 
clear signals that decisions on resource alloca- 
tion should be in part based on formal iden- 
tification and (where possible) measurement 
and valuation of relevant costs and benefits. 


Should this approach be taken, Congress 
could, for example, inform HCFA and PHS that 
it approves of efforts examining the possibility 
of including cost effectiveness of medical tech- 
nology as a criterion for reimbursement cover- 
age and that it'approves use of such a criterion if 
that examination finds it to be feasible. Reim- 
bursement coverage is mentioned here only as 
an example; the difficulties in this area are sub- 
stantial (see Option 4) and this may not be an 
area where Congress wishes to encourage CEA/ 
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CBA. Similar statements can be made in regard 
to encouraging the use of CEA/CBA in market 
approval decisions by FDA. 


There are, however, areas where more explic- 
it consideration of costs and benefits might be 
encouraged with less negative impact. One is in 
resource allocation decisions by health planning 
agencies. Costs and benefits are already listed as 
criteria for such decisions. Also, quality assur- 
ance activities of PSROs might be modified to 
take costs and benefits into more explicit consid- 
eration. Encouragement in these areas might 
take the form of providing increased analytical 
capabilities to agency staffs, providing technical 
assistance based on modified CEA/CBA-de- 
rived national data for agency use, and, im- 
portantly, signaling a congressional desire that 
all relevant costs and benefits of agency deci- 
sions be considered, not just those costs or ben- 
efits that may be easily quantified. 


The essence of this option is that costs and 
benefits should be a part of resource allocation 
decisions, but that current methodologies argue 
against leaning on formal analysis too heavily. 


A positive aspect of this option is its direct 
recognition of the necessity of having some way 
of making resource tradeoffs. The option may 
also result in an increase in knowledge about the 
value of current and, eventually improved, ana- 
lytical techniques such as CEA/CBA. A nega- 
tive aspect is the difficulty of encouraging use of 
CEA/CBA while not requiring it. Substantial 
thought should go into the specific means of im- 
plementing this option in individual health care 
programs. 


OPTION 4 
Specify that medicare reimbursement criteria 
should include consideration of technology’s 
cost effectiveness. 


This option would add consideration of costs, 
which have not been a criterion to date, to the 
largest Federal health program. The option’s de- 
sirability depends on philosophical, methodo- 
logical, and logistical factors. Philosophically, 
selection of this option would signal a change in 
the rationale of medicare and perhaps other 


Government-funded health programs. Current- 
ly, medicare operates, with some exceptions, as 
a program to enable the aged and disabled to 
have access to the medical services available to 
the general population. For some very expensive 
life-prolonging technologies (e.g., renal dial- 
ysis), however, medicare has become the vehicle 
by which those technologies are made generally 
available to all age groups. 


If, under this option, cost-effectiveness cri- 
teria are applied to all technologies to be cov- 
ered under medicare, the aged and disabled 
might be denied reimbursement for use of tech- 
nologies that were still available to other in- 
sured populations. 


Further, major changes in the procedures for 
raising coverage questions would have to be 
made. The local contractors might have to refer 
a great many questions to HCFA for resolution, 
and HCFA would depend on PHS for a large 
volume of analysis. Moreover, the data on 
which coverage recommendations and decisions 
would have to be made do not exist for many 
technologies. The administrative expense would 
be large and the analytical base would have to 
be greatly expanded at considerable cost. The 
potential cost savings from more cost-effective 
use of technologies, if that were brought about 
by application of CEA/CBA, would have to be 
compared to the expenses generated by this op- 
tion. There is, however, no way to estimate con- 
fidently either the probability or the amount of 
medicare cost savings that might result. This op- 
tion, if implemented in the absence of an overall 
system to control expenditures, would have lit- 
tle effect by itself on the absolute amount of ex- 
penditures. 


Consideration might be given, however, to 
requiring that the possible addition of major 
high-expense technologies or inexpensive but 
high-volume technologies to medicare be care- 
fully assessed and their potential impact on the 
total health budget be analyzed. Tradeoffs be- 
tween increases in medicare expenses and in- 
creases in medicaid benefits or PHS programs 
could then be considered. 


OPTION 5 
Require that resource allocation decisions by 
health planning agencies be based in part on for- 
mal cost-effectiveness criteria. 


This option implies a belief that the state-of- 
the-art of CEA/CBA is sufficiently advanced to 
provide useful and valid information to health 
planners. It further implies that State and local 
health planning communities and their constitu- 
ents will accept the results of CEA/CBA studies 
as inputs to the decision process. The feasibility 
of the option also depends on health planning 
agencies’ developing the capability to perform 
or commission adequate analyses or having ac- 
cess to studies done by others that will be adapt- 
able to the local situations. 


OTA finds that, although recent legislation 
does indicate that Congress intends that health 
benefits be weighed with costs in planning deci- 
sions and that HRA is providing some guidance 
to State and local planning agencies in this 
regard, there is no consensus as to an adequate 
analytical method to which health planners can 
turn. At the same time, however, OTA finds 
that helpful information for many decisions can 
be obtained by following generally accepted 
principles of analysis, which include explicitly 
enumerating all the costs and benefits of a given 
course of action. In general, health planning 
agencies at present do not have sufficient techni- 
cal skills to perform formal CEA-type studies. 


OPTION 6 
Modify the food and drug laws to include cost 
effectiveness as a market approval criterion for 
drugs and medical devices. 


This option might force FDA to formally 
compare the safety, efficacy, and costs of a new 
product with those same characteristics of ex- 
isting technologies. Such an effort would re- 
quire an extensive amount of data, much of 
which does not currently exist. The analytical 
capabilities of FDA would have to expand and 
change markedly to incorporate the new crite- 
rion. The administrative and analytical de- 
mands of this option would be enormous. 


Most importantly, the decision of whether to 
select this option would hinge on current con- 
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gressional intent regarding market approval. 
Past intent, as reflected in the statutes, explicitly 
mentions only safety and effectiveness (effi- 
cacy). Approving this option would mean that 
the intent of Congress regarding FDA’s regula- 
tory function has changed. If such is not the 
case, the option is inadvisable. If intent has 
changed, then the feasibility of the option de- 
pends on the minimization of the administrative 
and methodological problems. 


Options Addressing the Techniques of 
CEA/CBA Themselves, or Their 
Development 


OPTION 7 
Encourage research on the methods of CEA and 
CBA, concentrating on general principles of 
analysis. 


OPTION 8 
Encourage the conduct of increased numbers of 
CEA/CBAs of medical technology. 


OPTION 9 
Encourage the development of a strategy for 
identifying and collecting information needed 


for CEA/CBA. 


OPTION 10 
Require or encourage demonstration projects to 
test the feasibility of CEA/CBA as an aid in 
decisionmaking. 


These options are not mutually exclusive; any 
combination of them could be implemented. 
They are designed to yield information about 
the future usefulness of CEA/CBA in decision- 
making and to increase CEA/CBA’s potential 
for being useful. None of the options requires an 
immediate commitment to or decision on the ul- 
timate use of CEA/CBA. All might contribute 
to advancement of the state-of-the-art of 


GEA/CBA.: 


Research as proposed in Option 7 might con- 
centrate on issues such as the development of: 
1) principles of formal analysis, as presented 
above, that could be used by analysts to con- 
duct studies and by decisionmakers to evaluate 
and use studies; 2) “array” or nonaggregated 
methods of presenting and considering costs and 
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benefits; 3) measurement indexes that attempt 
to capture and weigh divergent variables; and 
4) analytical approaches to various categories of 
decisions in health care, for example, the use of 
diagnostic technology, or assessment of new 
versus established technologies. This research 
could be coordinated with the studies that 
would be conducted under Option 8. Those 
studies, in order to help determine the potential 
usefulness of CEA/CBA, should be undertaken 
or at least designed and evaluated in collabora- 
tion with agencies making policy decisions relat- 
ing to medical technology. Option 8 cannot be 
merely an academic exercise; it cannot be done 
in a policy vacuum. If it is, few relevant lessons 
about CEA/CBA will result. Further, because 
the number of technologies that could be studied 
is extremely large, the setting of priorities for se- 
lecting those to be analyzed is of critical impor- 
tance. 


Option 9 does not mean the collection of ad- 
ditional data. OTA believes that such collection 
should be postponed until much more thought 
has gone into a strategy that specifies the most 
useful kinds and forms of information needed to 
conduct analyses of costs and benefits. The use- 
fulness of CEA/CBA is critically dependent on 
the quality of the data available. Currently, the 
state of many types of data, especially data 
on efficacy and safety of medical technology, is 
inadequate. Much work has already gone into 


examining the state of health data systems and 
content. The strategy envisioned by Option 9 
would build upon the existing studies, but 
would require consideration of the specific 
needs of CEA/CBA analysts and decisionmak- 
ers. 


The need for some method of comparing the 
effects of health care activities with the re- 
sources consumed will remain critical to policy- 
makers. Thus, the potential of CEA/CBA or 
some derivation should be explored fully. This 
may require, or at least benefit from, limited ex- 
periments on the actual formal application of 
CEA/CBA in program decisionmaking. Three 
possible areas for the demonstration projects of 
Option 10 may be medicare’s reimbursement 
coverage decisions, the resource allocation ac- 
tivities of health planning agencies, and the re- 
view activities of the PSROs. Option 10 recog- 
nizes that the methods of CEA/CBA may im- 
prove, that data inadequacies may be lessened, 
and that methods of assuring the appropriate 
consideration of CEA/CBA results (in relation 
to other variables of the decision) may be devel- 
oped. Thus, today’s judgments of the role and 
usefulness of CEA/CBA may need modification 
later. In fact, demonstration projects may help 
to advance the usefulness of CEA/CBA by con- 
tributing to advances in methods, data, and so 
on. 


2. 
Evaluating Costs and 
Benefits in Health Care 
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2. 


Evaluating Costs and 


Benefits in Health Care 





INTRODUCTION 


A substantial share of the health policy com- 
munity seems to perceive cost-effectiveness 
analysis/cost-benefit analysis (CEA/CBA) as a 
potentially significant aid in the attempt to re- 
duce health cost inflation and inefficient re- 
source allocation. Officials of both public and 
private sector third-party payers have cited 
their desire for CEA/CBA to assist them in re- 
imbursement decisions. Recent conferences on 
medical technology issues, attended by experts 
from academia, Government, and the private 
sector, have focused on the need for technology 
assessment, given its presumed significant po- 
tential role in reimbursement decisionmaking 
(427). In addition, the Federal Government is 
building the institutional machinery to guide, 
support, and possibly use analysis. 


More generally, the health care literature ex- 
hibits a growing interest in the cost effectiveness 
of health care.1 Further, a wide variety of health 
care organizations have demonstrated concern 
with the resource allocation and cost contain- 
ment issues that CEA/CBA is presumed to ad- 
dress. Illustrative of the interest and concern are 
growth in the teaching of health economics in 
medical school curricula (18), the separate ef- 
forts of the American College of Physicians and 
the Resident Physicians Section of the American 
Medical Association to promote cost-effective 
care, and the work of Professional Standards 
Review Organizations, medical society cost 
containment committees, and so on. 


The potential of CEA/CBA to contribute sig- 
nificantly to cost containment and improved re- 
source allocation seems to be an article of faith 
to many officials and health policy experts, but 
both the potential significance and nature of any 

1See Background Paper #1: Methodological Issues and Litera- 


ture Review, prepared by OTA in conjunction with this assess- 
ment. 


contributions of these techniques remain to be 
established. Despite the fact that different 
observers have different kinds of impacts in 
mind when they discuss these analytical tech- 
niques, the several qualitatively distinct kinds of 
impacts CEA/CBA might have are rarely articu- 
lated explicitly. 


The purpose of this chapter is to set the stage 
for later discussions of findings regarding the 
usefulness or potential usefulness of CEA/CBA. 


The public, or governmental,” sector is called 
upon to make certain decisions that are imprac- 
tical for the private sector to make. Examples of 
these are decisions concerning national defense 
or air pollution control, neither of which is 
amenable to being traded in the marketplace. 
The public sector also must make other deci- 
sions for social reasons such as assuring equit- 
able distribution of what may be considered es- 
sential goods (e.g., health care of the elderly). 
Although the inherent complexities of these de- 
cisions and the uncertainties of the variables in- 
volved make it extremely difficult to identify 
and weigh all the consequences, the quality and 
validity of the decisions can be greatly enhanced 
by structuring the process so as to reveal as 
many of the relevant costs and benefits as is fea- 
sible and by evaluating them from a social per- 
spective. 


Since conventional private sector techniques 
such as capital budgeting and return-on-invest- 
ment analysis are ordinarily insufficient for de- 
cisions made in the public arena, special tech- 


2This report uses the term “public sector” to refer to governmen- 
tal institutions and the term “private sector” to refer to all other 
segments of society. Some people further distinguish between the 
for-profit “private” sector and the not-for-profit “private” sector 
(including, e.g., foundations, universities, or institutions such as 
the Blue Cross Association). The term “independent sector” has 
been suggested for this latter sector. 
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niques have been developed. In the medical care 
area, two such techniques are CEA and CBA. 
CEA and CBA are designed to integrate the eco- 
nomic aspects of a decision with the health as- 


DEFINITION OF CEA/CBA 


The terms CEA and CBA have come to refer 
to formal analytical techniques for comparing 
the positive and negative consequences of alter- 
native ways to allocate resources. In practice, 
the comparison of costs and benefits is accom- 
plished through a spectrum of approaches, 
ranging from sophisticated computer-based 
mathematical programing using large amounts 
of epidemiological and other data to partially 
intuitive, best-guess estimates of costs and bene- 
fits. Some analyses may take into account the 
results of clinical trials of a technology and 
model the technology's effect on health out- 
comes. Others may assume that the alternative 
technologies under study have equal effective- 
ness and concentrate on the difference in costs 
involved. 


Thus, there is a continuum of analyses that 
examine costs and benefits. At one end of the 
continuum are what will be referred to as “net 
cost” studies. In these studies, the emphasis is on 
costs; net cost studies in the past have often as- 
sumed benefits or efficacy to be equal. At the 
other end of the continuum are analyses that at- 
tempt to relate the use of the technologies under 
study to specific health-related outcomes and 
compare the costs of the technologies to the dif- 
ferential health benefits. Thus, CEA/CBA com- 
prises a set of analytical techniques, differenti- 
ated by the specific costs and benefits that are 
considered and the manner in which they are 
analyzed. 


The principal distinctions between CEA and 
CBA lie: 1) in the method of valuation of the 
desirable consequences of a decision, and 2) in 
the implications of the different methods of that 
valuation. In CBA, all costs and all benefits are 
valued in monetary terms. In a CBA (simplified 
for illustration) of an emergency medical sys- 
tem, for example, the cost of the program would 


pects of that decision. Consequently, they 
should not be considered simply as economic 
tools. 


be estimated and compared to the benefits. The 
latter, which might consist of saving a certain 
number of lives, would somehow be valued in 
dollars, and the results of the analysis would be 
expressed in dollar cost per dollar benefit. Thus, 
conceptually, CBA can be used to evaluate the 
“worth” of a project and would allow compari- 
son of projects of different types (e.g., such as 
dams and hospitals). 


In CEA, on the other hand, desirable program 
consequences are not valued in monetary terms, 
but rather are measured in some other unit; 
common measures include years of life saved, 
days of morbidity or disability avoided, or com- 
bination of morbidity and mortality such as 
quality-adjusted life years (QALYs) saved. The 
reason for a nonmonetary measure of program 
effectiveness is either the impossibility or the 
undesirability of valuing certain outcomes in 
dollars. Thus, unlike that of a CBA, the “bot- 
tom line” of a CEA is not expressed as a net 
monetary value or a monetary ratio; rather, it is 
expressed in units such as “dollars per life 
saved” or “dollars per QALY gained.” CEA does 
permit comparison of cost per unit of effective- 
ness among competing program alternatives de- 
signed to serve the same basic purpose. Unlike 
CBA, however, it does not allow comparison of 
programs having vastly different objectives (be- 
cause the effectiveness or outcome measures dif- 
fer), nor does it permit assessment of the inher- 
ent worth of a program. Is a cost of $50,000 per 
year of life saved acceptable? Obviously, this 
question requires a social and political judg- 
ment; it is not simply a technical matter. Natu- 
rally enough, as the state-of-the-art of devel- 
oping composite indexes of outcome measures 
improves, the versatility of CEA also advances, 
because the technique can then be used to com- 
pare increasingly divergent programs. 


In this report, the term CEA/CBA is used to 
refer to the class of techniques that includes both 
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CEA and CBA. The findings of the following 
chapters apply generally to both techniques. 


IMPORTANCE OF EFFICACY AND SAFETY DATA 


As emphasized above, CEA/CBA should not 
be considered only an economic tool. This point 
is not negated by the fact that CEA/CBA is de- 
scribed as an efficiency-based technique. Meas- 
urement of efficiency depends as much on out- 
put as on resources used to produce the output. 
OTA believes that one of the critical output or 
outcome measures that are or can be addressed 
by CEA/CBAs is effect on health status or other 
health-outcome-related effects. Any CEA/CBA 


HISTORY OF CEA/CBA 


The commonsense principles of CEA/CBA 
have been promoted for centuries, but formal 
application of CEA/CBA is primarily a phe- 
nomenon of the present century. In 1902, the 
River and Harbor Act directed the Corps of En- 
gineers to assess the costs and benefits of river 
and harbor projects. In 1936, the Federal Flood 
Control Act required that “the benefit (of proj- 
ects) to whomsoever they may accrue (must be) 
in excess of the estimated costs,” though the Act 
provided no guidance as to how benefit and 
costs were to be defined and measured. In the 
same decade, both the Tennessee Valley Au- 
thority and the Department of Agriculture im- 
plemented program budgeting systems that in- 
cluded rudimentary attempts at formal CEA/ 
CBA. Official Government criteria for appraisal 
of river development projects were first enunci- 
ated by the Bureau of the Budget in 1952 (585). 


Early in the Kennedy administration, the De- 
fense Department, under Secretary McNamara, 
adopted a program budgeting system that em- 
ployed CEA/CBA to evaluate alternative de- 
fense projects. Success in these endeavors, com- 
bined with a burgeoning Federal budget, led 
President Johnson in 1965 to require the imple- 
mentation of planning-programing-budgeting 
(PPB) systems throughout the Federal Govern- 
ment. CEA/CBA represented both the spirit and 


that attempts to analyze such outcomes for eval- 

uation of medical technology will be only as 
comprehensive and valid as the data on the ef- 

ficacy and safety of that technology are. Thus, 

health-outcome-related CEA/CBAs are depend- 

ent on the existence of an adequate efficacy and 

safety information base. As OTA has reported 

previously, however, the status of such infor- 

mation is inadequate for many medical technol- 

ogies (465). 


the letter of the new initiative to rationalize 
Government resource allocation decisionmak- 
ing (547). 


PPB met with mixed and limited success, re- 
flecting a lack of resources to implement it effec- 
tively, political and bureaucratic opposition to 
it, and unrealistic expectations of its role and 
potential (256, 352). The formal system did not 
survive for long, though many Washington ob- 
servers believe it left a legacy of continuing im- 
provement in the use of rational analysis in 
Government decisionmaking (516). The philo- 
sophy and logic of CEA/CBA and PPB have 
been reincarnated during the Carter administra- 
tion in the form of zero-based budgeting. 


In this assessment, OTA found relatively few 
sound applications of CEA/CBA in health re- 
source decisionmaking. There are, however, in- 
creasing numbers of such studies being per- 
formed, particularly in the applied research 
field, and the results of these studies are increas- 
ingly being disseminated. Although it is difficult 
to know how much effect this type of informa- 
tion has, there is ample evidence that both the 
private and public health sectors are increasing- 
ly cost conscious.? 


3See ch. 2 of Background Paper #1: Methodological Issues and 
Literature Review, prepared by OTA. 
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OTA found two divergent trends in published 
CEA/CBA studies. One trend, based largely 
within the academic community but slowly 
diffusing to research-oriented practitioners, is 
leading to rapid changes in the state-of-the-art 
of CEA/CBA. This trend is toward CEA/CBA 
studies characterized as interdisciplinary, of 
high quality, and advancing the state-of-the-art 
of the methods. These studies are also becoming 
increasingly oriented to concerns peculiar to the 
health field, such as health status measurement, 
equity considerations, and the value of health 
and of life itself. Since the studies are gaining 
wide readership—being published in the leading 
journals—and are becoming more understand- 
able to the lay public, they are enhancing ac- 
ceptance of CEA/CBA. 


LEGAL STATUS OF CEA/CBA 


Currently, the law explicitly authorizes only 
one health-care-related agency, the National 
Center for Health Care Technology, to support 
CEA/CBA studies, although parts of the Na- 
tional Health Planning and Resource Develop- 
ment Act (Public Law 93-641) require that cost 
effectiveness be considered in some decision 
processes. Furthermore, no court rulings or 
pending cases directly relate to the use of these 
techniques in health care agencies. In areas 


The other trend, which is occurring outside 
the academic community, is characterized by a 
rapidly expanding CEA/CBA literature base. 
Many of the articles are written by practitioners 
who are increasingly concerned about the gene- 
ral concept of cost effectiveness in medical prac- 
tice. For the most part, this body of literature is 
found in general health and medical journals. 
Although the analyses are not as methodologi- 
cally advanced or complex as those in the form- 
er group—probably because the authors often 
do not have an economic or other quantitative 
background—their impact upon physician prac- 
tice may be substantial. The increasing number 
of studies, in any case, certainly can be consid- 
ered an index of practitioners’ concerns about 
health costs. 


other than health (e.g., environmental regula- 
tion), however, there are immediate and signifi- 
cant judicial and legislative pressures to use 
CEA/CBA in decisionmaking. In the last two 
Congresses alone, more than 65 separate bills 
have included provisions for Federal agencies to 
use CEA/CBA or risk-benefit analysis in the de- 
cisionmaking process. See appendix E for a brief 


discussion of the legal status of CEA/CBA. 


USE OF CEA/CBA IN HEALTH CARE DECISIONMAKING 


There are two basic types of health care re- 
source allocation decisions that in theory could 
benefit from CEA/CBA. The first are decisions 
made within a fixed, constrained, or popula- 
tion-based budget, such as those made by health 
maintenance organizations (HMOs). The sec- 
ond are decisions made in the absence of a direct 
budget constraint, such as reimbursement deci- 
sions by medicare or certificate-of-need recom- 
mendations by local health planning agencies. 


In the former—allocation decisions made 
within a budget—tradeoffs must be made, since 
not all projects can be funded. The projects that 
promise to deliver more benefits for the cost 


should be more attractive than those projects 
expected to deliver fewer benefits. In these deci- 
sions, an economic constraint already forces 
costs to be considered. The function of CEA/ 
CBA in budget-constrained decisions, therefore, 
would be to illuminate the decision process and 
to require that implicit judgments be made more 
explicitly, thus forcing external examination of 
assumptions and of the values placed on the de- 
cision variables. Note that even though costs are 
normally taken into account in these budget- 
constrained situations, the types of CEA/CBA 
or related analyses undertaken can still range 
from analyses on the net cost end of the con- 
tinuum to analyses where effectiveness is ex- 


plicitly related to health outcomes or some 
equivalent measure. 


In non-budget-constrained decisions, direct 
tradeoffs between competing projects often are 
unnecessary. Consequently, a function of CEA/ 
CBA in these decisions would be to force eco- 
nomic factors to be considered. In health plan- 
ning decisions, for example, planners would be 
asked not only to consider whether a service is 
needed but also to compare the cost of the serv- 
ice with the expected benefits and perhaps to 
compare the costs and benefits of the service 
under study to the costs and benefits of other 
services that could be assigned higher or lower 
priorities. 


CEA/CBA is viewed by different parties as 
ranging in usefulness from obfuscating the perti- 
nent issues in a decision process at one extreme 
to illuminating and synthesizing the issues so 
well that the technique is used to make decisions 
at the other extreme. There is, however, a mid- 
dle position that maintains that the technique 
could be helpful in structuring information and 
that this information should be only one of 
several components of a decision process. Both 
extreme positions mentioned above are associ- 
ated with the use of CEA/CBA as a formal, 
structured analysis that is oriented toward a 
bottom-line answer, such as a cost-benefit ratio. 
Such a bottom-line, however, may avoid or 
even hide many important value judgments, 
thus providing an unambiguous answer which 
may rest on ambiguous data or assumptions. 


Ch. 2—Evaluating Costs and Benefits in Health Care © 21 


Advocates of the middle position propose that 
CEA/CBA be used within the context of ac- 
cepted principles of analysis in order to il- 
luminate the costs and the benefits of a decision, 
but not necessarily to aggregate and weigh 
them. Warner mentions a similar perception of 
CEA/CBA as a consciousness-raising exercise: 
CEA/CBA would have “no direct influence on 
policy decisions, but its presence in the litera- 
ture and in policy debates (would serve) to raise 
the general awareness and understanding of the 
economic side of health care, particularly 
among members of the medical profession” 
(615). When properly conducted, CEA/CBA 
can serve as a means of raising value and equity 
issues related to the subjects under study. 


Most of the specific findings of this report re- 
late to two major general findings of the OTA 
assessment. The first of the general findings is 
that performing an analysis of costs and benefits 
has the potential to be very helpful to decision- 
makers, because the process of analysis struc- 
tures the problem, allows an open consideration 
of all relevant effects of a decision, and forces 
the explicit treatment of key assumptions. The 
second general finding is that CEA/CBA ex- 
hibits too many methodological and other limi- 
tations to justify relying solely or too heavily on 
the results of formal CEA/CBA studies in mak- 
ing a decision. Thus, CEA/CBA could be useful 
for assisting in many decisions, but is probably 
not appropriate as the sole or prime determinant 
of a decision. 
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3. 


Methodological Findings and Principles 


INTRODUCTION 


OTA found no consensus among analysts and 
practitioners as to a standard set of methods 
for cost-effective analysis/cost-benefit analysis 
(CEA/CBA). Although there is still some dis- 
agreement as to which variables should be con- 
sidered and how these variables should be 
treated—problems which may be lessened as the 
state-of-the-art develops—there is agreement 
that at present no one method is appropriate for 
any two classes of technologies or for any two 
situations under which a technology is being as- 
sessed. In general, the disagreement on precise 
methods is due more to the inherent nature of 
the analysis, the nature and stage of develop- 
ment of the technology being analyzed, and the 
general social and political environment of deci- 
sionmaking than to the immaturity of CEA/ 
CBA methods. OTA believes that the funda- 
mental approach to CEA/CBA should be based 
on clear, logical thinking, using explicit criteria 
within the framework of generally accepted 
methodological principles. 


METHODOLOGICAL LIMITATIONS 


The methodological weaknesses or shortcom- 
ings of CEA/CBA are of two general types: 1) 
those that are inherent in this form of analysis, 
and 2) those that are due to the lack of maturity 
in the state-of-the-art of CEA/CBA and to the 
lack of analyst expertise and experience with 
CEA/CBA in health care. The latter type can be 
expected to diminish as more experience accu- 
mulates. The 10 principles for analysis pre- 
sented later in this chapter are directly relevant 
to lessening this type of shortcoming, which will 
be called “weaknesses due to immaturity.” Limi- 
tations of the first type, those that are inherent, 
however, are likely to remain significant bar- 
riers to advances in the usefulness of CEA/CBA 
in health care. 


66-220 0 - 80 - 3 


OTA did find full agreement on the paucity 
of, and consequently the need for, improved 
data, without which good analyses are impossi- 
ble. Efficacy and safety information for many 
technologies is generally not available. Health 
care utilization data are often either not avail- 
able or not in standard format or accessible 
form, and cost data are often inaccurate and 
also nonstandardized. Better routine data col- 
lection, although desirable and possibly neces- 
sary, would probably not be sufficient for better 
analyses, however, because each specific anal- 
ysis often requires a unique data set that will not 
be available in even the best of routine data col- 
lection systems. Consequently, an optimum mix 
of routine data collection and study-specific 
data collection needs to be defined, and when 
specific studies are funded, monies for necessary 
data collection should be included. 


Weaknesses Inherent in CEA/CBA 


Inherent weaknesses of the methodology of 
CEA/CBA,! especially in its more formal or so- 
phisticated forms, may prove to be of greater 
long-run consequence than those of immaturity 
(617). 


Examples of such weaknesses are the difficul- 
ty of predicting with precision the costs and 
benefits of new or not yet existing programs or 
technologies, fundamental problems in quanti- 
fying or valuing certain important but less tan- 


1Inherent methodological weaknesses of CEA/CBA are dis- 


cussed in detail in Background Paper #1: Methodological Issues 
and Literature Review, prepared by OTA in conjunction with this 
assessment. 
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gible health benefits, controversy over the ap- 
propriate discount rate, the inability of analysis 
to adequately incorporate equity and political 
considerations, and the inevitability of signifi- 
cant sensitivities or uncertainties even in many 
perfectly managed studies. The rapidity and 
profundity of technical change in medicine exac- 
erbate analytical difficulties, a problem that is 
felt particularly acutely because the point at 
which an analysis might have the most signifi- 
cant impact on health resource allocation—be- 
fore a technology has diffused into widespread 
medical practice—is also the point at which 
evaluation uncertainties are most dramatic. 
Sensitivity analysis sometimes can demonstrate 
that inherent technical analytical problems do 
not affect qualitative conclusions, but frequent- 
ly these difficulties preclude a definitive assess- 
ment of the desirability of competing programs. 
Ultimately, research may resolve some current- 
ly intractable problems, but for the foreseeable 
future, most such limitations seem likely to re- 
main inherent barriers to the direct application 
of the findings of many CEA/CBAs to policy 
decisionmaking. In particular, the uncertainties 
which pervade analysis severely restrict the po- 
tential of a study, however high quality, to re- 
solve definitively the ‘close calls” in which alter- 
native programs are similar in both cost and ef- 
fectiveness. 


Weaknesses Due to Immaturity 


As noted in the previous chapter, there are 
relatively few examples of technically high- 
quality CEA/CBA studies in the health litera- 
ture. As the state-of-the-art of CEA/CBA ma- 
tures and as analysts and decisionmakers gain 
more experience with CEA/CBA in health care, 
however, there may be a reduction in the num- 
ber of problems such as inappropriate or inaccu- 
rate specification of production relationships;? 
inadequate identification, measurement, or val- 
uation of costs or benefits; lack of discounting 
of future costs and benefits; and failure to exam- 
ine sensitivities. Though one should never 
downplay the difficulty of producing a techni- 
cally high-quality study, in principle problems 
such as these can be resolved; clearly the prac- 
tice of analysis can and should improve over 
time. Thus, the current restriction on the useful- 
ness of CEA/CBA caused by weaknesses of im- 
maturity seems likely to recede in importance 
over time. The 10 principles of analysis pre- 
sented below are suggested as one method of 
minimizing these weaknesses. 


2See Background Paper #1: Methodological Issues and Litera- 
ture Review. 


TEN PRINCIPLES OF CEA/CBA METHODOLOGY 


There is widespread agreement that the 10 
basic principles below are generally applicable 


to CEA/CBA analysis. (See table 1.) 


1. Define Problem 


The problem should be clearly and explicitly 
defined and the relationship to health outcome 
or health status should be stated. The problem, 
for example, may be expressed ix terms such as 
“excess infection rate” or “excess deaths.” The 
broader the definition of the problem, the more 
relevant alternatives there are to examine. For 
instance, “excess deaths” could lead to compar- 
ing any preventive or therapeutic program that 
decreases mortality, whereas “excess deaths due 
to cancer” would limit the scope of study con- 


siderably, and “excess deaths due to cervical 
cancer” would limit it even further. Neverthe- 
less, whatever the scope, as long as the focus is 
on a health problem, the study can focus on al- 
ternative means to solve the problem or, con- 
versely, to increase or improve health status. 
Some studies, however, must necessarily focus 
on the efficient use of a technology. This is par- 
ticularly true of studies of diagnostic technol- 
ogies, where the ultimate health problem may 
be far removed from the use of the technology. 


2. State Objectives 


The objectives of the technology being as- 
sessed should be explicitly stated, and the anal- 
ysis should address the degree to which the ob- 


Table 1.—Ten General Principles of Analysis 
(for CEA/CBA methodology) 





Define problem. 

State objectives. 

Identify alternatives. 

. Analyze benefits/effects. 

. Analyze costs. 

. Differentiate perspective of analysis. | 
. Perform discounting. | 


. Analyze uncertainties. 


OMAN OA FwWhND = 


. Address ethical issues. 


are 
oO 


. Interpret results. 





SOURCE: Office of Technology Assessment. 


jectives are (expected to be) met. In general, the 
objectives will be governed by the way in which 
the problem is defined; the broader the problem 
definition, the broader the objectives. Ordinari- 
ly, it is most relevant for the objectives to be in 
terms of lowering morbidity, disability, or mor- 
tality or, alternatively, increasing well-being. 
When the objectives are stated in terms of de- 
creasing costs, the relationship between costs 
and health benefits is often lost, sometimes re- 
sulting in untenable assumptions of equal ef- 
ficacy across treatment modalities. Often, ob- 
jectives are stated in terms of achieving a certain 
level of benefit for the least cost, or, conversely, 
achieving the most benefit per dollar cost. 


3. Identify Alternatives 


Alternative means (technologies) to accom- 
plish the objectives should be identified and sub- 
jected to analysis. The number of alternatives 
and the relevancy of the analysis will increase as 
the scope of the identified problem is increased. 
Whereas there are numerous means to lower 
death rates, for example, there are relatively 
fewer ways to lower deaths due to a specific 
disease, and even fewer ways to lower these 
deaths by employing a particular technology. 
One of the most difficult questions to answer in 
analyzing the cost effectiveness of a given inter- 
vention (such as Pap screening) is ‘cost effective 
compared to what?” 
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4. Analyze Benefits/Effects 


All foreseeable benefits/effects should be 
identified and when possible should be meas- 
ured. The relevant effects of health care technol- 
ogy in the health field often follow directly from 
the problem under consideration, the objectives 
specified, and the framework in which the prob- 
lem is approached. Not all benefits or effects are 
positive—some may be negative (e.g., deaths 
due to surgery) and some may be indeterminate 
(e.g., incurable disease may be discovered). 
Each of the following categories should be con- 
sidered: 1) personal benefits/effects, such as 
alleviated pain, reduced risk of sickness or 
death, enhanced quality of life, lowered anxiety, 
2) health resource benefits/effects such as in- 
creases and decreases in health care expendi- 
tures, 3) other economic benefits/effects such as 
increased productivity, and 4) social benefits/ 
effects such as the equitable distribution of med- 
ical care. When possible, and if agreement can 
be reached, it is helpful to value benefits in com- 
mon terms in order to make comparisons across 
alternative programs easier. 


5. Analyze Costs 


All expected costs should be identified and 
when possible should be measured in dollars. In 
general, the concept of “opportunity cost” is the 
most correct way to consider the costs of a pro- 
gram. That is, the costs are equal to the value of 
the opportunities that are forgone because of the 
investment in the program. 


6. Differentiate Perspective of Analysis 


When private benefits and costs differ sub- 
stantially from social benefits and costs, and if a 
private perspective is appropriate for the anal- 
ysis, the differences should be identified. Al- 
though CEA/CBA is generally considered a tool 
of social policy, it is helpful and important to 
recognize that private incentives differ from 
public incentives and since health care delivery 
is often funded, always demanded, and usually 
delivered by the private sector, its (the private 
sector's) perspective may be very important to 
the relevancy of the analysis. For instance, the 
social benefits of elective procedures such as 
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elective hysterectomy, cancer screening, and 
many psychotherapy programs are apt to differ 
markedly from the private benefits. Typically, a 
CEA will identify the “social” benefits in terms 
of cost reduction; the primary private objective 
(i.e., expected benefits) of the patient, however, 
may be decreased anxiety. 


7. Perform Discounting 


All future costs and benefits should be dis- 
counted to their present value in order for them 
to be compared with one another. Discounting 
can be thought of as a reverse interest rate. It is 
used to take into account phenomena such as 
the observation that, all things being equal, peo- 
ple prefer benefits (including health benefits) to- 
day rather than at a future time. 


8. Analyze Uncertainties 


Key variables should be analyzed as to the 
importance of their uncertainty to the results of 
the analysis. That is, a “sensitivity analysis” 
should be performed. In its simplest form, sen- 
sitivity analysis is nothing more nor less than 
the application of commonsense when one is not 
sure of a fact; it is the examination of the uncer- 
tain event under different assumptions. Sensi- 
tivity analysis can indicate both when more in- 
formation is needed and when insufficient infor- 
mation is irrelevant. 


OTHER FINDINGS 


In addition to conforming to the aforemen- 
tioned 10 general principles, all quantitative 
analyses should specify data sources, be written 
as clearly and as nontechnically as possible, and 
be subjected to peer and other types of review, 
including public scrutiny when appropriate, 
especially regarding assumptions upon which 
the outcome of the analyses may rest. In gen- 
eral, the more technical the analysis, the more 
important that the review be formalized and 
conducted by individuals who can challenge the 
methodology that is employed. Reviews of 
those CEA/CBAs that are not too technical, 
however, may facilitate public scrutiny regard- 


9. Address Ethical Issues 


Ethical issues should be identified, discussed, 
and placed in appropriate perspective relative to 
the rest of the analysis and the objectives of the 
technology. Many health care programs have as 
their primary objective the equitable distribu- 
tion of services; other programs include it as one 
of many objectives; still other programs affect 
the distribution of society’s goods and services 
without an explicit intention to do so. A CEA/ 
CBA should identify all these effects. When pos- 
sible, it should also measure them. Although 
such effects cannot ordinarily be valued, how- 
ever, they are often germane, and sometimes 
essential, to the measure of worth of a health 
program. 


10. Interpret Results 


The results of the analysis should be discussed 
in terms of validity, sensitivity to changes in 
assumptions, and implications for policy of de- 
cisionmaking. This is important both because 
the intended audience is often a public official or 
a health care professional, neither of whom may 
be technically oriented, and because study find- 
ings are often reported in capsule form such as a 
news brief, and are often introduced in the pro- 
fessional literature in abstract form. Results of 
CEA/CBA often have the potential to mislead 
the reader, a hazard which can be greatly re- 
duced by interpretation. 


ing the validity and, especially, the appropriate- 
ness of key assumptions. Such scrutiny may be 
useful because the application of CEA/CBA in 
the field of health policy is only part of a larger 
political process. 


Since this report is primarily designed to ex- 
amine the policy implications of using CEA/ 
CBA for health care resource allocation deci- 
sions, the methodological process which is envi- 
sioned is substantially different from what 
would be discussed if this report were being 
written for the academic research community. 
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It is necessary to make this distinction be- 
cause CEA/CBA can be a very complex under- 
taking analytically and often requires a massive 
data-gathering effort. For instance, disease pro- 
gression rates must often be assigned and math- 
ematical models must capture the dynamics of 
the process; the effects of medical intervention 
may need to be estimated by professional opin- 
ion or empirically evaluated through epidemio- 
logical observation or by formal clinical trials; 
joint production costs may need to be estimated 
using sophisticated dynamic programing tech- 
niques; and so forth. All this is expensive, time 
consuming, and is apt to require very special- 
ized computer support, analytical skills, and 
clinical judgment. On the other hand, the real 
world dictates that health resource allocation 
decisions must often be made without the bene- 
fit of such resources—that is, with little time, 
money, and technical expertise. These subopti- 
mal conditions, however, do not relieve deci- 
sionmakers from the responsibility of weighing 
the consequences of decisions. 


Since CEA/CBA is being spoken of or advo- 
cated as a mechanism to assist policymakers in 
making rational choices between competing ob- 
jectives, OTA was asked to assess the technique 
for that purpose. The findings are that, as for- 
mally applied, this analytical method could 
often be too complex, expensive, and time con- 
suming if used as a routine method for decisions 
by public policymakers. In fact, the cost-effec- 
tiveness case studies conducted as part of this 
assessment serve to highlight the immaturity of 
the technique itself.? Initial drafts of more than 


3See Background Paper #2: Case Studies of Medical Technol- 
ogies, Background Paper #3: The Efficacy and Cost Effectiveness 


NONAGGREGATED ANALYSIS—AN 


Since many of the methodological weaknesses 
of CEA/CBA may be hidden, aggravated, or in 
fact caused by the practice of deriving a cost- 
benefit or cost-effectiveness ratio—that is, a 
numerical bottom-line—the possibility of not 
aggregating the often complex sets of calcula- 
tions should be investigated and considered. In- 
stead of aggregating, analyses might be done by 


half of the studies, all of which were performed 
by respected health care researchers, were con- 
sidered by reviewers to be inadequate with re- 
spect to the relevancy/usefulness of the results, 
the validity of the methods, the tenuousness (or 
error) in the key assumptions, or the validity of 
the data used. Clearly, the field is not yet fully 
defined. 


Nevertheless, the logic behind using CEA/ 
CBA, even at an operational or policymaking 
level, appears sufficient to suggest that the 10 
principles previously enumerated can and 
should be followed under most circumstances. 


In no way, however, does this finding suggest 
that a complete analysis is either easy or unnec- 
essary. There is clearly a need for ongoing and 
sophisticated studies of the cost effectiveness of 
specific technologies as well as a need for ad- 
vancing the state-of-the-art itself. For instance, 
much good research has been done in develop- 
ing and testing sets of indexes that describe the 
health status of a population at any given time 
(79). That type of work should continue and 
perhaps should receive more emphasis. Never- 
theless, formal CEA/CBAs, however poten- 
tially valid and effective, can be inappropriately 
used by decisionmakers who lack the necessary 
resources and skills. Defining a more practical, 
limited approach to the methods seems clearly 
appropriate and does not diminish the worth of 
or need for more sophisticated approaches un- 
der different circumstances. 


of Psychotherapy, and Background Paper #5: Assessment of Four 
Common X-Ray Procedures, prepared by OTA in conjunction 
with this assessment. 


ARRAYING TECHNIQUE 


explicitly listing or ARRAYING all the elements 
that are included in, or would be affected by, 
decisions. 


When costs and effects can be quantified, that 
would be done; when they can be combined, 
that would also be done. Whenever one or more 
important nonquantifiable variables would 
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otherwise either be left out or be relegated to a 
footnote, however, no effort to arrive at a single 
combined benefit value would be made. 


A nonaggregated or array method of analysis 
would give decisionmakers a greater number of 
elements to consider, but it would also make in- 
tangible or nonquantifiable factors more expli- 
cit, and thus might help force consideration of 


these factors by decisionmakers commensurate 
with the factors’ significance. 


A more detailed examination of this arraying 
possibility,along with a discussion of circum- 
stances leading to OTA’s suggesting it, is found 
in Background Paper #1: Methodological Issues 
and Literature Review. 


4. 
Uses and Usefulness of 
CEA/CBA: General Findings 
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4. 


Uses and Usefulness of 
CEA/CBA: General Findings 


INTRODUCTION 


Substantial disagreement and confusion sur- 
round the question of the potential usefulness 
of cost-effectiveness analysis/cost-benefit anal- 
ysis (CEA/CBA) in decisions regarding medical 
technology and the health care system. With the 
continuing concern over health care expendi- 
tures, and with the advocacy of CEA/CBA by 
many people and groups as a means to amelio- 
rate cost-related problems, this confusion and 
disagreement take on a significance that is far 
more than academic. 


OTA believes that the potential usefulness of 
CEA/CBA depends very critically not only on 
the feasibility but also on the implications of its 
use. Accordingly, in this assessment, OTA ex- 
amined three major issues: 1) the general use- 
fulness (past and potential) of CEA/CBA in de- 
cisionmaking regarding medical technology; 
2) the methodological strengths and limitations 
of CEA/CBA; and 3) the potential for initiating 
or expanding the use of CEA/CBA in health 
care decisionmaking regarding medical tech- 
nology, especially in six health care programs— 
reimbursement coverage, health planning, mar- 
ket approval for drugs and devices, Profession- 
al Standards Review Organizations (PSROs), 
R&D activities, and health maintenance organi- 
zations (HMOs). A major aspect of the exami- 
nation of the third issue is the potential impli- 
cations—to the programs, to decisionmaking 
quality, and to society’s values—of CEA/CBA 
use in the six program areas. 


The primary focus of this assessment is on the 
application of CEA/CBA to medical technolo- 
gy—which OTA defines as the drugs, devices, 


and medical and surgical procedures used in 
medical care, and the organizational and sup- 
port systems within which such care is pro- 
vided. With the exception of a background pa- 
per on CEA/CBA and psychotherapy,’ the as- 
sessment does not directly address psychosocial 
medicine. Other aspects of health, such as the 
environment, are also not covered. OTA be- 
lieves, however, and it was the consensus of the 
advisory panel, that the findings presented in 
this report, and in the background paper on 
methodological issues, may apply also to other 
areas such as health care resource allocation in 
general. With modification, the findings may 
also apply to areas such as environmental health 
regulation, occupational safety and health, and 
education resource allocation. 


Furthermore, although the subject of the as- 
sessment was CEA/CBA, the findings should 
also be examined with an eye to their applicabil- 
ity to other types of formal analysis. Risk-ben- 
efit analysis, decision analysis, systems anal- 
ysis, technology assessment, and social impact 
assessment, for example, are all techniques used 
to examine various policy questions in both 
public and private organizations. The usefulness 
and implications of each of these techniques will 
vary according to many of the same factors that 
affect the usefulness of CEA/CBA. In fact, there 
are only hazy distinctions between these other 
forms of analysis and the forms of CEA/CBA. 


‘Background Paper #3: The Efficacy and Cost Effectiveness of 
Psychotherapy, prepared by OTA. 

*Background Paper #1: Methodological Issues and Literature 
Review, prepared by OTA. 
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GENERAL FINDINGS 


OTA found few examples of well-conducted, 
sophisticated CEA/CBAs conducted for and 
used in decisionmaking in health care. It is like- 
ly, however, that the extent of use of CEA/ 
CBAs in health care decisionmaking OTA 
found in its survey (see app. B) understates ac- 
tual usage—of informal CEAs in particular, but 
of formal, relatively sophisticated analyses, as 
well. OTA’s survey was not exhaustive. The ef- 
fort that was undertaken to ascertain the 
amount of use, though, does seem to indicate 
that the level of use is not substantial. Use of 
formal CEA/CBA in decisionmaking in health 
care is the exception not the rule. 


It is safe to say, however, that most decisions 
made take into account only a subset of the po- 
tential consequences of those decisions. The in- 
herent complexities of many decisions and the 
uncertainties of decision variables make it ex- 
tremely difficult to identify and weigh all the 
consequences. In general, OTA found, the qual- 
ity and validity of decisions can be increased by 
analysis that forces a structuring of the decision 
process, that provides a framework for identify- 
ing and considering as many of the relevant 
costs and benefits as is feasible. 


This finding supports the two major general 
findings of the assessment that were presented at 
the end of chapter 2. The process of CEA/CBA 
may be more helpful generally than would be 
the rigid and formal application of CEA/CBA 
study results in health care programs. 


Chapter 2 also set out two broad classes 
of health care program decisions: constrained 
budget ones, and nonbudget or nonconstrained 
ones. CEA/CBA potentially can be more valua- 
ble for decisionmaking under a constrained 
budget where tradeoffs have to be made directly 
than when constraints are nonexistent or very 
indirect. Under the budget system, the budget it- 
self would act as a cost containing or controlling 
factor. Under the nonconstrained type of sys- 
tem, since no direct tradeoffs are required, no 
direct limit on expenditures is set or forced. 
Thus, in neither case would CEA/CBA neces- 
sarily function as an effective cost-constraining 
mechanism or tool. Advocacy of CEA/CBA as 


such a tool, therefore, should be regarded skep- 
tically. CEA/CBA might, though, change the 
mix of expenditures. Technologies might be sub- 
stituted for one another on the basis or partially 
on the basis of analysis—especially under a 
budget situation. In this regard, there is poten- 
tial for CEA/CBA to help increase efficiency, 
even in terms of health outcome, without neces- 
sarily lowering total expenditures. 


Related to this last point about the possible 
use of CEA/CBA in improving resource alloca- 
tion within a constrained budget is the observa- 
tion that this country’s health care system might 
move closer to an overall constrained budget. 
This is mentioned only as a possibility. Perhaps 
in the future, health care expenditures may be 
limited (or constrained) to a fixed or specified 
percentage of gross national product, or to some 
specified absolute amount of dollars. If this situ- 
ation ever comes about, or even as an increasing 
number of individual institutions and programs 
operate under budget constraints, the appropri- 
ateness of CEA/CBA may increase. In such a 
possible future situation, most resource alloca- 
tion decisions would require explicit tradeoffs. 
It is conceivable, therefore, that efforts devoted 
to the development of a CEA/CBA-based ap- 
proach to decisionmaking (not necessarily tied 
to numerical study results) will represent an 
investment in future social policymaking. The 
lack of direct applicability of formal CEA/CBA 
to many of today’s decisions may then be offset 
by future applications of CEA/CBA. 


Various sources consulted and numerous peo- 
ple interviewed by OTA for this assessment pro- 
vided information yielding several seeming con- 
tradictions or paradoxes concerning CEA/CBA. 
For example, one common argument is that use 
of CEA/CBA may often be unethical if it does - 
not take values and distributional issues into ac- 
count adequately. There is validity to that state- 
ment. But there also seems to be validity to the 
argument that not considering costs and benefits 
in decisions on society's resources, especially in 
an area so basic as health, is unethical, because 
in the absence of the explicit consideration of 
consequences and of the parties on whom those 


consequences may fall, inequities will very like- 
ly occur. 


Another example of a seeming paradox con- 
cerns the “power” of CEA/CBA results. Some 
people argued that because many decisions are 
made in a political context, the results of any 
“objective” analysis would be heavily criticized 
and overwhelmed by other factors. Yet others 
argue that one of the factors in the potential 
misuse of CEA/CBA is its quantitative nature, 
allowing those involved in the decision process 
to “anchor” their arguments to what appear to 
be hard numbers. Are the results of CEA/CBA 


powerless? Or overly powerful? 


The resolution of both these examples may lie 
in the distinction between the process or ap- 
proach of CEA/CBA and the quantitative re- 
sults of formal studies. As indicated by the two 
general findings of this assessment, many of the 
negative perceptions of CEA/CBA are based on 
the possible misuse or inappropriate use of for- 
mal study results. Viewed as a method of struc- 
turing the decision process, CEA/CBA need not 
hide or avoid questions of ethics or values, and 
it need not provide a deceptively quantitative 
answer to complex problems. 


As an example of the difficulty of concentrat- 
ing on quantifiable variables and how investiga- 
tions of decision possibilities might be enhanced 
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by thinking in CEA/CBA terms, consider the 
cost effectiveness of CEA/CBA itself. OTA was 
frequently asked whether a CEA/CBA of CEA/ 
CBA might not be what is needed. And for a 
given decision situation that type of analysis 
might be very valuable. Approaching a CEA/ 
CBA of CEA/CBA in order to arrive at a quan- 
titative, traditional bottom-line result, how- 
ever, might lead analysts to list as a primary 
“cost” of CEA/CBA the resource costs involved 
in conducting and interpreting the studies. 
Thus, resource costs such as those identified in 
appendix C would be included, with dollars be- 
ing the measure used. If, however, the analysts 
were less interested in a bottom-line figure for 
the CEA/CBA of CEA/CBA, they might con- 
sider the opportunity costs of analyses. That is, 
the more important aspect of the costs of CEA/ 
CBA may not be the dollars it takes to conduct 
it, but rather the alternate uses of those dollars 
and the alternate types of analysis and other ac- 
tivities that might occupy the attention of those 
concerned about more rational allocation of 
medical technologies (617). Would the funds 
and attention that could be devoted to CEA/ 
CBA be more productive if applied to efficacy 
and safety studies? To education or conscious- 
ness-raising of physicians? To more dissemina- 
tion of existing knowledge of the costs and bene- 
fits of various technologies? To regulation of the 


use of technology? These are the questions that 
probably should be asked. 


FACTORS AFFECTING THE USE AND USEFULNESS OF CEA/CBA 


One of the key factors affecting the uses and 
usefulness of CEA/CBA has already been dis- 
cussed in chapter 3: the technical, methodologi- 
cal feasibility of the technique. These methodo- 
logical factors can be inherent aspects of CEA/ 
CBA, or they can be due to the state-of-the-art 
of CEA/CBA and thus more tractable. 


The manner in which both types affect the 
usefulness of CEA/CBA, however, should be 
analyzed in the context or the environment of 
current or potential uses of CEA/CBA. In other 
words, the questions should be asked what is the 
decisionmaking context and how does it affect 


the strengths or limitations of the methodology, 
and vice versa? For example, does the decision 
relate to a technology at an early stage in its life- 
cycle, such as bone marrow transplants? Or 
does it concern an established technology, such 
as appendectomy? Is the technology in question 
a diagnostic technology, such as the CT scan- 
ner, or a therapeutic one, such as renal dialysis? 


The possibility of affecting the course of a 
technology's diffusion and use might be greater 
in early stages of its development, but the uncer- 
tainties about its health effects and its costs will 
generally be greater. Thus, it may be possible to 
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do a more valid or certain CEA/CBA later in 
the technology’s lifecycle, but the information 
gained may be less valuable for public policy. 
The tradeoff required will vary depending on 
the specifics of the technology and the policy de- 
cision to be made. In addition, diagnostic tech- 
nologies are often more difficult to study than 
other technologies because of the uncertainties 
involved in linking their use to health outcomes. 
Thus, studies of diagnostic technologies often 
tend toward the “net cost’ end of the CEA/CBA 
spectrum, where the measures of outcome or 
benefit may be numbers of tests performed or 
levels of diagnostic accuracy. 


In sum, the stage of development of the tech- 
nology under study and the type of technology 
(or function of the technology) are two of the 
factors that will affect the specifics of analysis to 
be used, the uses to which analysis can be put, 
and the usefulness of resultant information. 
Other factors are the relative strength or im- 
portance of nonanalytical factors, such as poli- 
tics or equity, in the decisions to be made; the 
ability of the sponsors of analysis to implement 
the results; the familiarity of sponsors and deci- 
sionmakers with formal analysis; the existence 
of adequate data relating to the technology, to 
the disease or other problem addressed by the 
technology, or on other possible effects of inter- 
ventions based on analysis; the existence of eco- 
nomic incentives that match or run counter to 
the results of analysis, the types of decisions to 
be made (e.g., budget-based decisions or non- 
budget-based decisions); and so on. Some of the 
factors that affect the use of CEA/CBA are 
listed in table 2. 


One of the factors listed above is of particular 
importance: the quality and availability of data. 
Obviously, without data or estimates of data, 
there would be no CEA/CBAs. The quality of a 
CEA/CBA is directly related to the accuracy of 
the data used in it. For example, when good epi- 
demiological data on the effects of a technology 
or the existence of disease are present, analysis 
will have a greater potential for being relevant 
and useful. A specific example of where epi- 
demiological data have permitted analyses of 
high quality is in the area of smoking and its ef- 
fects on health. Good data do not guarantee 
good analyses, however, because the quality of 


Table 2.—Factors Affecting the Use of CEA/CBA 


Stage of Development of the Technologies Under Study. 
—Tradeoff required between availability/validity of data 
and ability to affect the future use of the technologies. 
Both the type of analysis and the usefulness of analysis 
will be affected. 


Nature of Technologies Under Study and Function of Tech- 
nologies Under Study.—I!n terms of function, diagnostic 
technologies, for example, often have indirect connec- 
tions to health outcome and often lend themselves to the 
net cost type of CEA/CBA. In terms of the physical nature 
of technologies, surgery, for example, may involve addi- 
tional uncertainties due to varying skills of surgeons and 
surgical settings. Both type and use of analysis will be af- 
fected, but especially the type or specific methodological 
elements. 


Social, Ethical, or Value Influences in the Decision Environ- 
ment.—Very similar, often overlapping with the above 
factor. Will affect both the type and uses of the analysis. 
The example of renal dialysis applies here. Abortion 
would serve as another example. 


Quality of the Analysis.—Can be of at least four types: 

Analysis Subject to Inherent Methodological Limitations. 
—e.g., inability to adequately deal with equity con- 
cerns; influence of discount rate chosen on outcome 
of analysis. 

Analysis Subject to State-of-the-Art Limitations.—e.g., 
difficulties in identifying and measuring many costs or 
effects. 

Analysis Containing Errors of Omission or Commission. 
—These are errors not due to the state-of-the-art, e.g., 
failure to discount or perform sensitivity analysis when 
appropriate. 

Analysis Subject to Data Limitations.—This factor will af- 
fect quality even though the other factors might have 
been adequately dealt with. Much cost and health out- 
come data are uncertain, difficult to retrieve, or simply 
nonexistent. 

All four of these factors can affect the quality of analysis, 

which in turn affects the usefulness of the results. 


Ability of Sponsors or Users of Analysis to Implement Re- 
sults.—The usefulness of analysis will naturally depend 
on the amount of control the user has over the particular 
technology or situation studied. 


Experience/Familiarity of Users With the Type of Analysis 
Conducted.—This factor will affect usefulness in two 
ways: it will be a direct influence on the acceptability of 
results, and it will affect the ability of the users to appro- 
priately apply the results. 


Existence of Economic Incentives in the Decision Environ- 
ment.—If the economic incentives relating to the use of 
the technology under study are in accord with the results, 
their acceptability will be great. If they run counter to the 
results, the usefulness will be limited, depending on the 
strength of the economic incentives. 


SOURCE: Office of Technology Assessment. 


analysis is also affected by the other factors 
mentioned above. Similarly, the usefulness of 
analysis is dependent on those factors affecting 
quality as well as on a number of other factors 
(see table 2) relating to the decisionmaking and 
analytical contexts or environments. 








There are many gaps in the data available for 
CEA/CBA, owing to such factors as methodo- 
logical constraints, inadequate resources for 
data collection and interpretation, lack of com- 
munication between the users of data and those 
collecting it, and the sheer impossibility of col- 
lecting and analyzing all the data that could be 
used by someone, somewhere. The principal 


POTENTIAL USERS OF CEA/CBA 


Health care policies and other decisions are 
made at a variety of levels and in a variety of 
situations by an extremely broad range of indi- 
viduals and groups. In theory, CEA/CBA re- 
sults or approaches might be useful to any or all 
of these decisionmakers. Table 3 lists many of 
the decisionmakers—the list is not exhaustive 
but should provide an idea of how diverse and 
numerous the types of decisionmakers are. 
Three general classes of decisionmakers or po- 
tential users of CEA/CBA information are dis- 
cussed in this assessment: individual medical 


Table 3.—Partial List of Individuals and Groups 
Making or Influencing Resource Allocation 
Decisions 


Individual physicians and other health care professionals 

Individual patients 

Medical professional societies and boards 

Consumer groups 

Health industry representatives and organizations 

Hospitals, clinics, other health care institutions 

Labor organizations 

Businesses 

Health maintenance organizations 

Medicare and medicaid 

Other governmental health care programs 

Health systems agencies, State agencies 

Professional Standards Review Organizations 

Blue Cross and Blue Shield Associations 

Other health care insurers, third-party payers 

Other quality assurance or utilization review groups 

Food and Drug Administration 

Ratesetting commissions 

Voluntary health organizations 

Public health departments 

Other State and local health agencies 

U.S. Congress, executive agencies, State legislatures 

Health care systems, such as the Veterans Administration’s 
and the Department of Defense’s 

Medical schools 

Biomedical and health services researchers 

Other health-related associations 


SOURCE: Office of Technology Assessment. 
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Federal agency charged with collecting and ana- 
lyzing health data is the National Center for 
Health Statistics (NCHS). NCHS is currently in- 
volved in several developmental projects in- 
tended to clarify certain methodological issues 
related to the provision of data for CEA/CBA, 
especially in relation to cost-of-illness studies 
(see app. B). 


practitioners, nongovernmental institutions, 
and governmental/quasi-governmental institu- 
tions. 


Individual Medical Practitioners 


Despite the fairly small amount of empirical 
research on the subject, it seems safe to say that 
CEA/CBA has had little direct impact on indi- 
vidual physicians’ behavior.? 


Discussions with academic physicians indi- 
cate a consensus regarding the above point on 
CEA/CBA’s lack of impact. Beyond that point, 
however, the consensus dissolves. There is dis- 
agreement, for example, concerning whether 
CEA/CBA has, and if so the extent to which, 
significantly affected physicians’ consciousness 
of economic issues. Explanations for the lack of 
impact on practice are numerous, with emphasis 
on their relative importance varying dramatical- 
ly from one observer to the next. And the con- 
sensus on current practice impact does not 
translate into agreement on the future role of 
CEA/CBA in influencing individual physician 
behavior: Opinion seems to be split roughly in 
half between those who believe that CEA/CBA 
will cause many physicians to alter their medical 
practices and those who anticipate continuation 
of the current absence of significant effect. 


The principal explanations for CEA/CBA’s 
lack of impact on physicians’ behavior to date 
can be grouped under two headings: 


1. The novelty of CEA/CBA in health care. 
Until very recently, the literature on health care 


3The following discussion is taken from work done for OTA by 


Kenneth Warner of the University of Michigan (615). 
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CEA/CBA was sparse. As indicated in Back- 
ground Paper #1, this has been particularly true 
in the medical literature. Relatively few physi- 
cians read the nonmedical health care literature; 
hence their exposure to the concepts and prac- 
tice of CEA/CBA was minimal prior to the last 
few years. Needless to say, lack of exposure cor- 
related highly with (and presumably caused in 
part) a lack of understanding of the technique 
and meaning of CEA/CBA. 


The novelty of CEA/CBA in health care ac- 
counts for some of the quality problems in the 
published literature. While poor analytical qual- 
ity certainly could be a barrier to application of 
the results of analysis, few observers cite it as a 
significant factor in the failure of physicians to 
apply findings to their practices. 


In a similar vein, the uncertainties in analysis 
frequently prevent determination of an unequiv- 
ocal conclusion in an analysis. Even when a firm 
“bottom line” is presented, nonquantified fac- 
tors—for example, the distribution of costs and 
benefits—can make the conclusion far from de- 
finitive. Thus, one could argue that even high- 
quality analyses frequently do not produce find- 
ings that can or should be translated directly in- 
to practice by individual physician decision- 
makers. This seems an attractive explanation 
for physicians’ nonresponse to analysis, partic- 
ularly combined with whatever bewilderment 
they may feel as a result of their unfamiliarity 
with CEA/CBA. It is not, however, an explana- 
tion often noted in discussions on the subject. 
Most likely, this explanation presupposes that 
other, preliminary barriers to application of 
analysis have been surmounted; the evidence is 
to the contrary. Thus, one might anticipate that 
such inherent technical limitations of analysis 
will grow in importance as other barriers fall. 


2. The irrelevance of much of CEA/CBA to 
medical practice decisionmaking. There are two 
basic sources of irrelevance, one substantive, 
one structural. In the substantive category, 
many CEA/CBAs have involved assessments of 
the desirability of social programs where social, 
and not individual, decisionmaking was at 
issue. Examples include the several studies of 
communicable disease control programs (e.g., 
measles, swine flu, etc.), community (or indus- 


try) based screening programs (e.g., hyperten- 
sion), and fluoridation of municipal water sup- 
plies. The subject matter of such studies pre- 
cludes a direct practice response by individual 
physicians. 


While this too may serve as a useful partial 
explanation of the absence of behavioral re- 
sponse by individual practitioners, it cannot ex- 
plain the total absence of such response, since 
much of the health care CEA/CBA literature is 
clearly relevant to individual practice decision- 
making. Nor is this an often-cited explanation. 
A more cogent argument concerns structural ir- 
relevance: According to a strict economic inter- 
pretation, most physicians’ interests in “cost- 
effective care” deviate significantly from those 
of society. All physicians share an interest in 
understanding the efficacy and safety of medical 
technologies—technologies whose risks out- 
weigh medical benefits are undesirable—but 
concerns with the economic side of cost effec- 
tiveness are either nonexistent or dependent on 
the physicians’ economic environment. In gen- 
eral (619): 


Cost data are psychologically remote. (The 
physician’s) one-on-one relationship with the 
patient is not in the context of the cost to socie- 
ty. 


The physician's economic circumstances, 
however, can produce in the physician an often 
subconscious reaction to costs. To a fee-for- 
service physician whose patients are well in- 
sured, the cost of a technology may be irrele- 
vant, at least immediately. If the physician 
works within the context of prepayment, how- 
ever, the professional concern with cost effec- 
tiveness begins to approach the social concern. 
In all cases, the patient's economic wherewithall 
often will be a major consideration: In an en- 
vironment of prepayment or adequate insurance 
coverage, high costs of technologies do not 
translate into direct economic burdens on pa- 
tients; hence the high costs are something of an 
abstraction to both the immediate patient and 
the physician. 


This economic interpretation—emphasized 
by many knowledgeable observers—attributes 
the lack of effect of CEA/CBA on medical prac- 
tice to its irrelevancy and even inconsistency 


with medical norms, irrespective of the quality 
or quantity of the literature. Accordingly, un- 
less the reimbursement system is changed, this 
argument suggests, the future will auger little 
change in the application of CEA/CBA to indi- 
vidual practice decisionmaking. According to 
this explanation, physicians’ nonresponse to 
CEA/CBA is not necessarily a reflection of 
physicians’ selfish monetary interests, or their 
indifference to economic considerations. Ra- 
ther, nonresponse to CEA/CBA perhaps reflects 
physicians’ fulfilling their roles as agents of their 
clients—patients. A physician’s major responsi- 
bility may be to weigh all the costs and benefits 
to the patient and to his or her medical prac- 
tice—i.e., the aggregate of all the patients of the 
physician. 


This argument is not an entirely economic 
one, because the ethics of the doctor-patient re- 
lationship are involved. If a patient is not 
harmed economically by performance of a cer- 
tain procedure, even though only a small medi- 
cal benefit might be expected, what are the 
ethics of the individual physician’s denying or 
recommending against the procedure in order to 
represent society's cost and benefit priorities? 
The differences between social and individual 
economic and ethical considerations constitute 
the only frequently advanced explanation for 
physicians’ nonresponse to CEA/CBA that does 
not imply a brighter future for the ability of 
analysis to alter individuals’ medical practice 
policies. Systemwide changes in the economic 
environment, such as growth in HMOs or major 
reimbursement reforms, might more closely 
aline the practice of medicine with the precepts 
of analysis. The strength of the explanation does 
not depend on lack of understanding of CEA/ 
CBA within the medical community; hence an- 
ticipated increases in familiarity with analysis 
need not promote the direct application of find- 
ings. Accordingly, barring external pressures, 
the economic incentives and ethical norms of 
medicine may very well continue to preclude 
widespread application by practitioners of the 
findings of health care CEA/CBAs, with the ex- 
ception of the “easy” cases in which one proce- 
dure is demonstrated to be both more effective 
and less costly than an alternative. 
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Nongovernmental Institutions 


A variety of nongovernmental institutions are 
potential consumers of CEA/CBAs. Insurers 
have a direct economic incentive to find and 
promote cost effectiveness in the provision of 
health care services; officials of major insurers, 
including Blue Cross/Blue Shield, have ex- 
pressed their interest in the development of 
more and better CEA/CBAs to assist them with 
reimbursement decisions (see ch. 5). In an era of 
increasing restrictions on reimbursement, hos- 
pitals’ interests in enhancing efficiency are ob- 
vious. HMOs also have a direct economic inter- 
est in cost-effective care: Greater efficiency 
translates into lower, more competitive mem- 
bership rates and/or higher incomes for member 
physicians. Large business firms and unions 
have several reasons to be interested in CEA/ 
CBA: Greater efficiency in the provision of 
medical services to employees implies lower 
business costs or room for negotiation of other 
fringe benefits; health promotion and disease 
prevention among workers may increase pro- 
ductivity and reduce other costs of disability 
and morbidity; and so on. As major financers of 
the costs of illness, each of these organizations 
has not only an interest in promoting cost-effec- 
tive care, but also has the market power to 
translate judgments concerning cost effective- 
ness into changes in health practice. 


Another group of nongovernmental institu- 
tions, not directly involved in the financing of 
care, is evidencing interest in CEA/CBA: pro- 
fessional associations. Among such groups are 
the Association of American Medical Colleges, 
the American College of Physicians, the Resi- 
dent Physicians section of the American Med- 
ical Association, and the cost-containment com- 
mittees of several State medical societies. In 
part, the interest of such groups reflects con- 
cerns about the social implications of inefficient 
medical resource allocation; in part, interest 
may reflect a perception that if the health care 
community does not control cost inflation, 
Government regulators may attempt to do the 
job for it. Regardless of the motivation, the 
demonstrated interest suggests a receptivity to 
information that CEA/CBA in theory can pro- 
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vide. This interest extends beyond the medical 
community. The American Dietetic Association 
has recently completed a study of the costs and 
benefits of nutrition care services; dentists have 
discussed the relative efficiency of alternative 
methods of preventing caries (78); and so on. 
Whether the efforts of such groups will ever 
translate into significant practice changes re- 
mains to be seen. But the “cost consciousness- 
raising” function of CEA/CBA seems well 
served by such efforts. 


To date, direct application of CEA/CBAs to 
institutional policies has been limited. Some 
organizations have identified themselves as be- 
ing in the market for specific analyses—for ex- 
ample, several businesses want to learn more 
about the costs and benefits of certain disease 
prevention/health promotion programs for em- 
ployees (e.g., control of alcoholism and drug 
abuse, hypertension screening and treatment, 
executive exercise programs, antismoking pro- 
grams)—and analysts have responded with 
CEA/CBAs tailored to the specific institutions’ 
needs (275). Recent policy decisions of other 
organizations reflect a CEA/CBA mode of 
thinking, though the decisions have not derived 
from formal CEA/CBAs. For example, the na- 
tional Blue Cross and Blue Shield Associations 
have recommended that member plans not re- 
imburse for institutionally standardized bat- 
teries of laboratory tests on admission to a hos- 
pital. 


The question remains whether, and if so how, 
nongovernmental institutions will use CEA/ 
CBAs. Certainly, by virtue of its ability to 
clarify issues and collect and organize informa- 
tion, CEA/CBA could assist planning and deci- 
sionmaking within many of these organizations. 
Some kinds of findings might lend themselves 
neatly to policy decisionmaking. For example, 
persuasive evidence that a certain diagnostic 
procedure is both more expensive and less ac- 
curate than an alternative procedure could serve 
as solid grounds for nonreimbursement or non- 
acquisition of the former. A large cost differen- 
tial between two equally effective procedures 
might also serve as support for a use-constrain- 
ing policy decision, though opposition might be 
substantial if significant elements of the medical 


community questioned the procedures’ equality 
of effectiveness. Indeed, whenever significant 
technical disagreement on efficacy exists within 
the medical community, CEA/CBAs seem un- 
likely to overcome opposition to the policies 
they might recommend, possibly barring the 
case of a truly dramatic cost difference. 


This point deserves emphasis because of a 
major implication: Clear-cut, unobjectionable 
CEA/CBA results probably will be an excep- 
tion, not the rule. Furthermore, they seem likely 
to reflect reasonably obvious differences be- 
tween the alternatives being studied. When a 
CEA/CBA is undertaken out of genuine interest 
in evaluating alternatives, without significant 
prior expectations as to the outcome of the anal- 
ysis, that outcome is less likely to be definitive. 
Competing professional opinions on technical 
issues (e.g., diagnostic accuracy) exacerbate the 
problem. Thus, definitive CEA/CBAs may sup- 
port policy decisions, but their potential to 
shape such decisions seems limited by technical 
and political factors. 


Governmental and 
Quasi-Governmental Institutions 


A limitation of the preceding discussions is 
that they deal with the decisionmakers as classes 
of decisionmakers (e.g., nongovernmental insti- 
tutions, not a specific institution). The discus- 
sion and arguments will vary according to indi- 
vidual circumstances. For this reason, and be- 
cause its mandate is related to Federal pro- 
grams, OTA analyzed the potential use of CEA/ 
CBA by several individual Federal programs. 


As an example of a reimbursement program, 
medicare is used, but Blue Cross and Blue Shield 
are discussed briefly (ch. 5). The drug and 
device market approval processes of the Food 
and Drug Administration are also covered (ch. 
8). There are two examples of programs that are 
federally sponsored, with national policies and 
administration to a degree, but primarily car- 
ried out by quasi-governmental organizations at 
the State and local levels: the health planning 
program (ch. 7) and the PSROs’ programs (ch. 
6). The usefulness of CEA/CBA to the health 
care R&D activities of the Federal Government 


is examined, using four Federal agencies as ex- 
amples (ch. 9). Finally, a federally promoted 
program that is carried out by private sector 
organizations, HMOs, is discussed (ch. 10). 


Each of these programs, or decision areas, 
uses somewhat different approaches to prob- 
lems and decisionmaking. Specific approaches 
have been developed to address divergent and 
diverse issues. Distinct mechanisms to analyze 
decisions have been evolved in the various pro- 
grams. Although informal and implicit analysis 
of costs and benefits seems to be a frequent 
aspect of policy formulation in most of the pro- 
grams, however, OTA found very little formal 
use of CEA/CBA. In several of the areas, one 
being market approval, cost itself has played lit- 
tle or no role in decisions. Figure 1 provides a 
view of the relationship of the six programs to 
each other and to the lifecycle of medical tech- 
nologies. Table 4 is a narrative explanation 
of that figure. An organization chart of the 
Department of Health and Human Services 
(DHHS), as seen from a perspective of interest 
in medical technology, is presented in figure 2 in 
order to show the organizations of chapters 6 
through 10 in relation to other elements of 


DHHS. 


The Federal Government is not the only pub- 
lic institution interested in analysis. Given med- 
icaid and other health care finance programs, 
States share the Federal Government's concern 
with health cost inflation. Local governments 
have also invested in analytical capability (see 
app. B). Indeed, one of the largest and most pro- 
ductive government analysis staffs was housed 
in the New York City Health Services Adminis- 
tration. 


To date, Government has been the principal 
consumer of analysis. Still, most observers of 
CEA/CBA consider Government's past use 
quite limited. One reviewer, however, has con- 
cluded that “there is evidence to suggest that 
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such studies have played an important role in 
public policy determination” (16). According to 
Dunlop, two examples support this conclusion. 
First, Enke’s mid-1960’'s finding that the benefits 
of birth control exceeded costs by a factor of 100 
contributed to Congress’ significant expansion 
of Agency for International Development funds 
to assist developing countries in implementing 
birth control programs. Second, Dunlop says, 
CBAs of water fluoridation have “nearly always 
preceded” dental health program development, 
with the studies being “widely disseminated to 
the public” prior to a public vote on fluorida- 
tion. 


Other analysts are less sanguine about the ef- 
fects of past analyses on policy. Jeffrey Weiss, 
who headed the Program Analysis staff of the 
New York City Health Services Administration, 
has concluded that, owing to political and budg- 
etary factors, his staff analyses had little impact 
on broad strategies. Analyses initiated by the 
staff lacked a political constituency and tended 
to be ignored. Of analyses requested by city 
officials, a few were followed by policies con- 
sistent with their findings, but these tended to 
support policymakers’ predetermined biases on 
the issues. Only when administrators had not 
worked through problems on their own, and 
strong political constituencies were not in- 
volved, could analyses affect the decisionmak- 
ing framework; and in most such cases, the ef- 
fects were “suboptimizing,” for example, resolv- 
ing narrow management issues. Weiss has cited 
a couple of successes, however—a critical anal- 
ysis of the neighborhood family care program 
resulted in constriction of the program; and a 
study of methadone treatment of heroin addicts 
created a strong intellectual rationale for drug 
detoxification efforts in New York City (355). 
Weiss noted that the city administrator might 
have gone ahead with the latter efforts in the ab- 
sence of the analysis, but the study provided 
support for the policy action (633). 
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Table 4.—Overview of Agency Activities in Decisions Concerning Medical Technology 





Decision areas and activities 


Agency/mission Financing 
1H 
Improve human health by 
increasing medical 
knowledge and encourag- 
ing the development of 
safe and effective med- 
ical technologies 











Supports and conducts 
basic and applied 
research 


Establishes standards for 
research, clinical trials, 
and human experimen- 
tation 

Supports and conducts 
clinical trials 

Sponsors Consensus De- 
velopment Conferences 
to evaluate medical 
technologies 

Evaluates outcomes of 

sponsored research 












































FDA 

Protect the American 
public from unsafe and 
inefficacious drugs and 
medical devices and un- 
safe foods and cos- 
metics 





Establishes regulatory requirements for demon- 
strations of the safety and efficacy of drugs and 
medical devices 

Evaluates safety and efficacy data from clinical trials, 
etc. 


Regulates market ap- 
proval of drugs and 
medical devices 

Regulates drug and med- 
ical device manufac- 
turing processes 

Monitors the safety of 

marketed drugs and 

medical devices 


NCHCT 

Undertake and support 
assessments of health 
care technologies 


Evaluates social, ethical, 
legal, medical, scientif- 
ic, and economic as- 
pects of new, emerging, 
and existing medical 
technologies 

Compiles and dissemi- 
nates information con- 
cerning the safety and 
efficacy of medical 

technologies 


Makes recommenda- 
tions concerning re- 
imbursement issues 
submitted by HCFA 


NCHSR 

Support health services 
research on a variety of 
health care issues 


HCFA 

Administer the medicare/ 
medicaid programs, 
PSRO program, and sup- 
port research and sta- 
tistics efforts for these 
programs 


HSAs? 

Develop and implement 
local health plans and 
monitor the dissemina- 
tion of health services 


Supports health services research, primarily on new 
and existing technologies 





Supports health services research on a variety of 
health care issues, primarily focused on areas of pro- 
grammatic responsibility 





Establishes reimburse- 
ment criteria for new 
and established medical 
technologies 


Establishes benefits 
packages for med- 
icare 

Makes reimbursement 
decisions concerning 
medical technologies 









Review major capital ex- 
penditures for cer- 
tificate of need 

Review use of Federal 

funds for certain pro- 

grams 


PSROs> 

Assure that health care Conduct research on Review and evaluate the appropriateness of health Support certain Fed- 
services paid for under quality of medical care care provided to Federal beneficiaries at acute care eral programs financ- 
certain Federal programs through medical care hospitals and long-term care facilities ing medical care 

are medically necessary, evaluation studies and 

meet professionally rec- profile analyses on 

ognized standards of physicians, patients, 

care, and are provided at and institutions 


the most economical lev- 
el possible consistent 
with quality care 








4At the national level, the Health Resources Administration’s Bureau of Health Planning is responsible for providing the community or regionally based HSAs with 
technical assistance and guidance regarding the planning and delivery of health care services. 
he Health Standards and Quality Bureau of HCFA, DHHS, is the Federal level organization that provides general guidance and technical assistance to the locally 
controlled and operated PSROs. 


SOURCE: Office of Technology Assessment. 
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Figure 2.— Department of Health and Human Services— Organizational Components Involved 
in Medical Technology 


__Secretary _ 
Under Secretary 


Asst. Secretary Asst. Secretary 


for Planning for Management 
& Evaluation & Budget 
Public Health Service Health Care Asst. Secretary for 
Asst. Secretary for Financing Human Development 
Health (OASH) Administration Services 
OHRST OHMO ODPHP 
NCHCT } | NCHS NCHSR 
(Public Health Service Agencies) 
ADAMHA CDC FDA HRA HSA MEDICARE] [MEDICAID] | HSQB 
NIAAA BoE BD BHP BMS OMAR PSRO 
NIDA BoL BMD BHF IHS NCI ESRD 
NIMH BHE BoB BHPr BCHS NHLBI 
NIOSH BRH NICHD 
Other NIAID 
Bureaus Other 
Institutes 


NOTE: See app. G for acronyms. 
SOURCE: Office of Technology Assessinent. 
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Reimbursement 


INTRODUCTION AND BACKGROUND 


Medicare and Blue Cross/Blue Shield, the 
largest government and private health insurance 
programs, respectively, pay for a substantial 
share of the Nation’s total health care outlay. In 
1978, for example, medicare financed 24 percent 
of all hospital care and 16 percent of all physi- 
cians’ services. In fact, its payments represented 
more than 15 percent of all personal medical ex- 
penditures in the United States. In the same 
year, Blue Cross/Blue Shield paid for nearly 
11.5 percent of all personal health care expendi- 
tures (84). Through financing, both programs 
can affect the rates at which new technologies 
are developed, diffused, and utilized, and at 
which inefficacious, outmoded, and unsafe serv- 
ices are phased out. 


This chapter focuses on the medicare program 
and its reimbursement coverage process. Also 
included is a discussion of Blue Cross and Blue 
Shield and its reimbursment coverage process. 
The latter discussion is included because the na- 
tional Blue Cross and Blue Shield Associations 


and the affiliated plans (the actual administra- 
tive units, who are relatively autonomous and 
who receive guidance but not mandatory proce- 
dures from the national associations) represent 
the largest nongovernmental third-party payer 
in the Nation and because their processes of de- 
ciding on coverage of medical technologies for 
reimbursement are similar to medicare’s. 


Medicare is a nationwide, federally adminis- 
tered health insurance program authorized in 
1965. It provides benefits for people over age 
65, for certain individuals eligible for disability 
payments, and for certain individuals who need 
kidney transplantation or dialysis. The medic- 
aid program is a Federal program that is admin- 
istered individually by each participating State 
government. Each State can use its own proce- 
dures for coverage decisions. Although medic- 
aid is not covered in this chapter, in concept, 
many of the arguments presented would apply 
equally well to that program. 


THE MEDICARE COVERAGE PROCESS 


Section 1962 of the Social Security Act man- 
dates that medicare shall pay only for medical 
services that are “reasonable and necessary” for 
diagnosis, treatment, or improved functioning. 
By granting Government officials the authority 
to determine which new and existing services? 
are eligible for reimbursement, this section of 
the law involves medicare in technology deci- 
sions. Medicare has refrained from establishing 
a definitive interpretation of the “reasonable 


1Sec. 1862.(a) Notwithstanding any other provisions of this title, 
no payment may be made under part A or part B for any expenses 
incurred for items or services (1) which are not reasonable and nec- 
essary for the diagnosis or treatment of illness or injury or to im- 
prove the functioning of a malformed body member. 

2The Social Security Act also specifically excludes certain serv- 
ices from payment eligibility. 


and necessary” language of the Social Security 
Act and relied on a loosely structured and de- 
centralized mechanism to provide coverage 
guidelines. 


The medicare program is administered in two 
parts: part A, financed by payroll taxes, pro- 
vides reimbursement for services in hospitals, 
extended care facilities, and other institutions; 
and part B, a voluntary supplementary pro- 
gram, pays for physicians’ services. Under the 
present coverage mechanism, the contractors 
who administer part A, fiscal intermediaries, 
and part B, carriers, of medicare bear the ini- 
tial responsibility for identifying coverage 
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issues and determining reimbursement policy.’ 
Through the advice provided by their medical 
advisors, contractors make coverage determina- 
tions about the majority of new services that 
they identify (98). 


When the contractors feel unable to make a 
judgment on a particular coverage issue, they 
submit the issue to one of 10 medicare regional 
offices. As a rule, referrals are made by carriers, 
who process physicians’ bills, rather than by 
fiscal intermediaries, who process hospital bills. 
This is because under the cost reimbursement 
system, hospital bills generally are not exam- 
ined for questions about the use of a particular 
technology. In fact, most hospital billing forms 
provide little specific information about the 
various elements of service (398,574). Although 
intermediaries, as well as hospitals, physicians, 
and the manufacturers of drugs and devices, 
may occasionally raise a coverage issue, carriers 
usually perform this function. 


Coverage decisions by both contractors (239) 
and regional offices (398) appear to be based 
primarily on two related criteria: 1) the tech- 
nology’s stage of development, and 2) its gener- 
al acceptance. If a new technology is perceived 
to have moved beyond experimental status 
toward full clinical application and to be ac- 
cepted by the local medical community, then it 
is deemed “reasonable and necessary.” These 
criteria, however, lack precise standards, and 
the contractors’ procedures for considering 
them tend to be informal and highly variable. 
Thus, for example, a medical advisor may base 
a decision on immediate personal knowledge of 
a technology’s stage of development and accept- 
ance or may survey the literature and seek out 
the opinions of other medical consultants, local 
specialty society representatives, advocates of 
the procedure, and the advisors of other area in- 
surance programs. 


In addition, both contractors and medicare 
regional offices appear to show considerable 
variation in the priority they accord to coverage 
questions and their approach to handling them. 
Some regional offices will attempt to resolve 


’Medicare contractors for part A deal with hospital services and 
contractors for part B deal with physicians’ services. 


many of the issues referred to them by seeking 
out the opinions of local contractors; others will 
tend to transmit such issues directly to the 
medicare central office (398,291). Similarly, 
within a region, some carriers will display far 
more initiative than others in identifying a 
coverage issue, pursuing information about it, 
and making a decision (398). As a result, the 
specific package of benefits for which medicare 
will provide reimbursement varies somewhat 
across the country, and there is no national 
standard for covered services. 


When the contractor and medicare regional 
office are unable to resolve an issue, it is re- 
ferred to the Health Care Financing Adminis- 
tration (HCFA).4 This agency, in turn, may re- 
quest a coverage recommendation® from the 
Public Health Service (PHS).° Historically, PHS 
generated its responses through an informal and 
loosely structured procedure. Typically, one, 
two, or three professional PHS staff members 
assigned to the task researched a coverage ques- 
tion by attempting to survey the relevant medi- 
cal literature and consult with appropriate ex- 
perts. This process has been inherently unsys- 
tematic, because the small PHS staff has lacked 
the benefit of established channels of communi- 
cations to medical specialty groups and to other 
PHS agencies such as the National Institutes of 
Health (NIH) and the Food and Drug Adminis- 
tration (FDA) (98). 


Within this ad hoc framework, PHS has tradi- 
tionally applied four criteria to coverage recom- 


‘Reimbursement coverage questions are referred to the Health 
Standards and Quality Bureau (HSQB) in HCFA. Prior to the crea- 
tion of HCFA, they were referred to medicare’s Division of Pro- 
vider and Reimbursement Policy. 

5In 1977, the Administrator of HCFA and the Assistant Secre- 
tary of Health entered into an agreement which formalized the 
Public Health Service's role in providing coverage recommenda- 
tions. Since the late 1960's, the medicare program’s lack of medical 
advisors has resulted in an almost automatic referral of coverage 
questions to PHS, although occasionally HCFA has tried to group 
related issues together or resolve a matter which seems to warrant 
little attention because it has either been previously raised and 
answered by PHS or it concerns an apparently worthless service 
(e.g., colonic irrigation) or a well-established and accepted one. 

*The current PHS unit providing coverage recommendations is 
the National Center for Health Care Technology (NCHCT). The 
Center was created by Public Law 95-623 in November 1978. Be- 
fore the formation of NCHCT, the Office of Health Practice As- 
sessment bore this responsibility. Previously, it was the function of 
the Bureau of Quality Assurance in the Health Services Adminis- 
tration. 


mendations: 1) safety, 2) efficacy, 3) stage of 
development, and 4) acceptance by the medical 
community. Although, these criteria have re- 
cently been made more explicit’? through a PHS 
staff paper on coverage policy,® they still lack 
precise definition. 


PHS recommendations generally have not at- 
tempted to recommend specific indications for 
use of technologies. The PHS staff have main- 
tained that coverage policies should be ex- 
pressed in broad terms and questions of appro- 
priate usage for a set of specific patient indica- 
tions should be addressed by the practicing 
medical community and the Professional Stand- 
ards Review Organizations. Thus, a PHS de- 
scription of its coverage policy notes (98): 


The coverage system cannot attempt to be an 
encyclopedic listing of which interventions to 
use in which circumstances nor serve as a kind of 
substitute for medical education and clinical 
training. Rather, the coverage system uses broad 
strokes to sketch the boundaries of accepted 
good medical practice, and leaves the fine-tun- 
ing of the system to the “back-end” mechanism, 
the Professional Standards Review Organiza- 


tions (PSROs). 


Recently, however, HCFA has shown some re- 
luctance in accepting PHS recommendations 
and sought to issue guidelines which cite indica- 
tions for use more often. 


After the PHS staff develop a coverage rec- 
ommendation, they send a memorandum to 
HCFA’s Health Standards and Quality Bureau 
(HSQB). This Bureau has usually accepted the 


7Historically, these criteria have been developed and utilized im- 
plicitly (33). 

8A PHS staff paper provides the elaboration of these terms (98): 

1. Safety: What is the likelihood and severity of risk associated 
with the technology? 

2. Efficacy: What is the likelihood that the technology will pro- 
duce the desired health benefit? No distinction is drawn here 
between efficacy and effectiveness, although in other con- 
texts, “efficacy” is used to mean the probability that a 
modality will produce a desired outcome under ideal circum- 
stances, while “effectiveness” is used to mean that the desired 
outcome will result under average clinical conditions. The 
term “efficacy” as used here is intended to convey a hybrid of 
both meanings, since the data base for coverage decisions is 
sometimes derived from laboratory or clinical trial environ- 
ments and sometimes from routine practice environments. 

3. Stage of development: Has the technology moved beyond 
the experimental stage to full clinical application? 

4. Acceptance by the medical community. 
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policy suggested by PHS and has issued appro- 
priate guidelines to medicare regional offices 
and contractors (282). Because of the large 
volume of services involved, policy implemen- 
tation depends mainly on the good faith of pro- 
viders and on the threat of potential audits. 


Cost and Efficacy 


Although the cost of medical technology has 
focused attention on reimbursement, cost infor- 
mation has not been explicitly or directly con- 
sidered in medicare coverage determinations. In 
a few cases, where PHS has concluded that a 
new technology is a modification of an existing 
intervention and offers little additional benefit, 
it has favored coverage but recommended pay- 
ment at the same level as that for the established 
service. This type of judgment represents a 
weighing of marginal cost and marginal value, 
so it might be said that PHS has conducted im- 
plicit cost-effectiveness analyses (CEAs). The 
practice of paying usual and customary fees, 
however, does not easily accommodate such 
analyses. 


Unlike cost, efficacy is one of the four criteria 
PHS has traditionally applied to coverage ques- 
tions. Several recent PHS procedural changes 
have promoted more systematic examination of 
efficacy data: 1) the development of communi- 
cation ties with NIH, FDA, and medical special- 
ty societies; 2) the formal linking of FDA drug 
and medical device policies to PHS coverage de- 
terminations (98); and 3) the setting of a prece- 
dent in the computed tomography (CT) body 
scan decision to restrict coverage to uses that are 
supported by current evidence of efficacy (282), 
although the list of indications for those scans 
appears to include nearly every possible use 
(389). 


The extent to which these procedural changes 
will ensure a greater emphasis on efficacy re- 
mains uncertain. Despite the CT body scan rec- 
ommendation, PHS has generally not attempted 
to specify indications for use, contending it 
would do so only in the consideration of drugs 
or in an exceptional case involving a high-risk 
or a high-cost technology. Although it is com- 
mitted to reflecting FDA's safety and efficacy 
rulings by following FDA's decisions on ap- 
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proved and contraindicated drug use, PHS exer- 
cises discretion in considering uses for indica- 
tions on which FDA has not ruled. Further, drug 
coverage questions are not usually raised, be- 
cause medicare only pays for the use of drugs in 
hospitals, and it is difficult for fiscal intermedi- 
aries to identify individual hospital service com- 
ponents.° Questions about medical equipment 
are far more typical, but the impact of relating 
coverage recommendations to FDA determina- 
tions is unclear, because the Medical Devices 
Amendments of 1976 are still in the process of 
being implemented (see ch. 8). Perhaps more 
importantly, adequate evidence of the efficacy 
and safety of devices and procedures is often not 
available (465), and in the absence of such evi- 
dence, technologies judged to be nonexperimen- 
tal and accepted usually have received a positive 
coverage recommendation from PHS. 


Application of efficacy criteria also is limited 
by the selected number of coverage questions 
that reach the PHS agenda.'° The current sys- 
tem, in which issues may be identified by fiscal 
intermediaries when they process hospitals’ bills 
and by carriers when they process physicians’ 
bills, is basically reactive. Identification of new 
services can be difficult, however, because in- 
formation may be lost or obscured not only by 
classification into service code categories, but 
also by translation from provider to intermedi- 
ary codes, although the tendency to attach a 
higher cost to a new technology serves as a ma- 
jor screening aid (399,569). Helen Smits, the Di- 
rector of HCFA’s HSQB has observed (575): 


A consistent method of coding diagnoses and 
procedures, consistently applied, is essential to 
any real “capture” of new technologies by reim- 
bursement . . . Even when a single procedural 
code has been agreed upon and put into use, 
problems on variable coding are likely to persist 
and to make accurate application of coverage 
decisions very difficult. 


Identification of a new hospital service is par- 
ticularly problematic. This is because, as noted 


‘It is also extremely difficult to monitor drug use for approved 
indications. 

tWhile judgments made by contractors and the regional office 
about acceptance and stage of development may be influenced by 
efficacy data, the focus appears to be more directly on stage of de- 
velopment and acceptance. 


above, fiscal intermediaries generally do not ex- 
amine hospital bills for coverage questions and 
the majority of billing forms! provide little in- 
formation about the use of particular technol- 
ogy. Certificate-of-need requirements, end of 
the year or prospective reimbursement negotia- 
tion, and informal discussions between inter- 
mediaries and the hospitals in their area appear 
to alleviate this problem only partially. 


Another way that a coverage issue may be 
raised is through the interaction of intermedi- 
aries and hospital administrators or staff when 
the hospital is planning to acquire or to offer a 
new service. Identification of large-scale, dis- 
crete, and potentially expensive technologies 
may be accomplished more easily through this 
type of interaction than it is through billings. 
Neither the extent nor the potential usefulness of 
such an identification method, however, was 


analyzed by OTA. 


Even when a new service is identified, the 
coverage decision often, perhaps usually, will 
be made by the contractor or intermediary and 
will not be brought to the attention of HCFA or 
PHS unless a negative determination is legally 
challenged by a physician, hospital, or patient. 
For example, PHS has never been asked to make 
a judgment about the coverage of coronary ar- 
tery bypass grafting (470), an expensive and 
widely performed procedure. In addition, the 
PHS coverage agenda is severely circumscribed 
by an almost exclusive emphasis on new tech- 
nology. The one major exception to this pattern 


occurred in 1977, when PHS issued recommen- 


dations about 28 established procedures after 
Blue Shield had concluded they were outmoded 
or ineffective and should be excluded from rou- 
tine reimbursement payments (431,432,433). 


Currently, however, both HCFA and PHS are 
studying a number of possible changes in the 
medicare coverage process that may increase the 


“Because of the diversity of elements involved in surgical proce- 
dures, identification of a “new” surgical procedure may be particu- 
larly difficult. For example, David Eddy notes, “In surgery, it can 
be very difficult to identify when a procedure is “new” or suffi- 
ciently different from other procedures to require a new evalua- 
tion . . . frequently a procedure can be described only as a set of 
maneuvers, and these maneuvers can change in subtle but impor- 
tant ways” (167). 


weight given to efficacy data, as well as formal- 
ly introduce cost criteria. HCFA is examining 
several possible actions: utilizing cost as a 
coverage criterion,’ implementing regulations 
that would formally define the “reasonable and 
necessary” language of the Social Security 
Act,}3 establishing a uniform service code,!4 and 
issuing more guidelines which relate coverage to 
appropriate indications for use.'* At the same 
time, the new PHS coverage-recommending 
unit, the National Center for Health Care Tech- 
nology (NCHCT) is considering utilizing three 
additional coverage criteria: conformity to 
health planning guidelines, relative efficacy, 
and cost effectiveness. Reservations have been 
raised about the introduction of the latter two 
criteria, however, because of methodological 
difficulties in measurement. As a result, the PHS 
coverage staff are currently awaiting the results 
of NCHCT’s experience with applying CEA in 
its comprehensive assessment activities. Ac- 
cording to a PHS staff paper (98): 


The chief difficulty is how to measure relative 
efficacy and cost-effectiveness in an operational 
way. If one is comparing the relative efficacy of 
two modalities, how much more efficacious 
must the more expensive or more risky technol- 
ogy be in order to be “worth” the added cost or 
risk? The techniques for comparing relative ef- 
ficacy and determining cost-effectiveness are less 
reliable than the methods for assessing safety 
and efficacy, and our experience in using the 
outputs of relative efficacy and cost-effective- 
ness studies as a basis for policy decisions is very 
weak. It would appear that some additional de- 
velopmental work is needed before relative ef- 
ficacy and cost-effectiveness can be applied 
routinely as criteria for coverage recommenda- 
tions. One way to demonstrate and test the ap- 
plication of these two criteria would be to use 
them in the course of the large-scale evaluations 


The Office of the General Counsel has been asked to determine 
whether the “reasonable and necessary” language of sec. 1862 and 
the reference to “reasonable cost” in other parts of the legislation 
permit introduction of cost criteria into coverage decisions (292). 

13HCFA's Division of Provider and Reimbursement Policy has 
drawn up a set of draft regulations defining “reasonable and nec- 
essary,” and the Office of Policy Planning and Research also is in 
the process of performing the same task. 

4A uniform code also may be used to produce a list of services 
that are covered (399). 

*8In recent months, HCFA has shown greater interest in issuing 
guidelines that cite indications for use. 
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to which NCHCT will subject the high priority 
technologies. If the “bugs” appear to have been 
worked out in the mega-assessments, considera- 
tion would then be given to applying relative ef- 
ficacy and cost-effectiveness more routinely in 
coverage decisions. 


Coverage Reevaluation 


Cost, safety, efficacy, and legal concerns 
have all contributed to the current reevaluation 
of medicare coverage decisions. Most impor- 
tant, perhaps, the rising cost of medical care has 
confronted policymakers with the need to con- 
tain health expenditures and rationally allocate 
resources. Increases in health care expenditures 
associated with both the enactment of medicare 
and the rapid diffusion and use of technology 
have generated additional interest in creating 
cost control mechanisms. 


Although the overall impact of technological 
innovation on health care spending is unclear 
(17), it is apparent that economic incentives 
strongly favor the spread of technology (196, 
528). Because medicare reimburses retrospec- 
tively, it provides an open-ended commitment 
to pay for covered services. Under medicare and 
some other third-party insurance, hospitals (the 
most expensive element of the health care sys- 
tem) are reimbursed on the basis of costs; physi- 
cians are reimbursed on the basis of charges; 
and patients are partly insulated from immedi- 
ate actual costs. As a result, services may be uti- 
lized even when patient outcome benefits are 
marginal or uncertain (468). 


The tendency toward utilization of services is 
also encouraged by other factors: competition 
among hospitals to achieve quality and prestige 
and attract patients and physicians; public de- 
mand for sophisticated technology; practition- 
ers’ desire to do the most possible for their pa- 
tients and to achieve a high degree of certainty 
in their judgments so as to avoid malpractice 
suits; physician specialization; and the stress on 
ancillary services (470,546). 


Spiraling medical expenditures and their asso- 
ciation with a reimbursement system that pro- 
motes technology development and use has en- 
couraged Government officials to reevaluate 
medicare’s traditional social insurance orienta- 
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tion. At the time of the enactment of the legisla- 
tion in 1965,'° and in the years since, there has 
been an emphasis on accommodation with the 
existing health care system (191).'” This theme is 
reflected in the nature of the present coverage 
mechanism: the heavy reliance on fiscal inter- 
mediaries and carriers, the failure to formally 
define the “reasonable and necessary” language 
of section 1862, and the tendency for PHS to 
react favorably to the majority of coverage 
questions it has addressed. But as cost contain- 
ment becomes an increasingly important objec- 


1eThe Social Security Act states in sec. 1801: 

Nothing in this title shall be construed to authorize any federal of- 
ficer or employee to exercise any supervision or control over the prac- 
tice of medicine or the manner in which medical services are pro- 
vided... 

For a political history of medicare, see R. Harris, A Sacred Trust 
(278); T. H. Marmor and J. Marmor, The Politics of Medicine 
(381), and O. W. Anderson, The Uneasy Equilibrium: Private and 
Public Financing of Health Services in the United States, “Uneasy, ” 
1875-1965 (25). 

Of course, the medicare legislation encouraged cost contain- 
ment through physician self-regulation by originally establishing 
hospital utilization review committees in 1965 and later creating 
PSROs in 1972. 


THE BLUE CROSS/BLUE SHIELD 


Cost considerations, along with safety and ef- 
ficacy and legal concerns, in addition to leading 
medicare to reevaluate its coverage policies, 
have led nongovernmental insurers to reevalu- 
ate their coverage policies. Traditionally, these 
insurers’ interest in escalating health care costs 
has been moderated by their ability to respond 
by raising insurance premiums. But as the pur- 
chasers of health insurance—management, la- 
bor unions, and individual subscribers—have 
become more resistant to higher premiums 
(470), third-party payers have been increasingly 
confronted with the need to contain health care 
costs and become more selective purchasers of 
health services. 


Blue Cross and Blue Shield, the largest non- 
Government insurers, with over 40 percent of 
the Nation’s subscribers, have called for cost 
containment initiatives since the early 1970's. 
Over the years, they have pursued a wide range 
of policies, including utilization review of length 


tive,1® interest in medicare’s becoming a more 
selective purchaser of health services is rising 
(34). Within this changing perspective, cost-ef- 
fectiveness information would seem to possess 
substantial appeal. 


Reexamination of the medicare coverage sys- 
tem also stems from a growing awareness that 
technological innovation and health status do 
not always seem to be directly related and that 
the safety and efficacy of many technologies 
have not been adequately evaluated (238). 
There is interest in protecting patients from 
risky, ineffective, or unproven services. The 
result has been that coverage decisions are 
becoming more closely tied to available safety 
and efficacy information. 


In addition, legal requirements have spurred 
PHS and HCFA review of the current coverage 
process. As the likelihood of more coverage de- 
nials increases, there is a greater need to estab- 
lish a firmer legal basis for decisions (291,574). 

18Feder suggests that reorganization was necessary for a shift in 


focus from social insurance to health policy (191). Several HCFA 
officials who were interviewed possessed a similar perspective. 


COVERAGE PROCESS 


of stay and level of care, health planning, bene- 
fit package designs that emphasize low-cost op- 
tions (such as outpatient surgery), alternative 
delivery systems, consumer education, and cov- 
erage reevaluation (415). Like medicare, how- 
ever, Blue Cross and Blue Shield traditionally 
have refrained from issuing many negative cov- 
erage guidelines. In part, this probably arises 
from a historical social insurance orientation, as 
well as Blue Cross and Blue Shield’s desire to: 1) 
remain competitive with commercial insurers, 
2) fulfill the expectations of beneficiaries, hospi- 
tals, and physicians that services will be covered 
by insurance policies, and 3) forestall legal chal- 
lenges that might result from denial of payment. 


Blue Cross and Blue Shield’s treatment of cov- 
erage issues parallels medicare’s in many re- 
spects. Like medicare, Blue Cross and Blue 
Shield make the majority of coverage decisions 
through a decentralized and loosely structured 
process that places key emphasis on two cover- 


age criteria: stage of development and accept- 
ance by the medical community. Blue Cross and 
Blue Shield also lack a uniform national benefit 
package; interpretations of what qualifies as a 
covered service vary somewhat throughout the 
country.?? 


Most individual Blue Cross and Blue Shield 
plan contracts exclude “care which is not,a part 
of generally accepted medical practice” and “un- 
necessary or inappropriate care,” although the 
specific wording employed in different contracts 
varies (255). These criteria lack precise defi- 
nition, however, so the medical advisors and 
their staffs possess considerable discretion in ap- 
plying them, although coverage questions that 
are viewed as particularly important may be 
brought to the attention of a plan's board of di- 
rectors. Medical advisors may make judgments 
based on immediate personal knowledge or may 
survey the literature and consult with advocates 
of the procedure, local specialty societies, the 
county medical society, and other area insurers. 
If the plan functions as an intermediary for 
medicare or an agent for medicaid or the armed 
services CHAMPUS program, the medical advi- 
sor will generally review the coverage determi- 
nations of these programs. Similarly, the medi- 
cal advisor will usually study the coverage rec- 
ommendations of the national Blue Cross and 
Blue Shield Associations and other member 
plans, although he or she may or may not fol- 
low any previous rulings. 


The national Blue Cross and Blue Shield As- 
sociations are a federation of 137 State and local 
nonprofit Blue Cross and Blue Shield affiliates. 
Power to make policy ultimately lies with the 
individual plans, but the national organization 
exercises considerable leverage through the 
prominence of its leadership and the ability to 
serve as a public spokesman for member plans 
(141,255,415). The national associations’ cover- 
age recommendations are made by the Medical 
Necessity Program and the Medical Advisory 
Committee. 


The Medical Necessity Program, developed in 
1977 in conjunction with the American College 


While benefit package coverage variation allows responsive- 
ness to local patterns of practice, it also raises the issue of equity. 
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of Physicians, the American College of Sur- 
geons, and the American College of Radiology, 
was designed to curtail reimbursement for out- 
moded, duplicative, or unproven procedures. 
According to a 1977 Blue Shield bulletin (433): 


The Program is an effort to align the objec- 
tives of several national professional medical so- 
cieties and of Blue Shield in improving the quali- 
ty of care while simultaneously reducing the 
costs of procedures that are, in many instances, 
outmoded, redundant in combination with 
others, unlikely to yield additional information 
through repetition, or of unproven value. 


Since the inception of this program, the Blue 
Cross and Blue Shield Associations have recom- 
mended discontinuation of routine payment for 
68 surgical and diagnostic procedures and hos- 
pital admission battery tests (58,59,140,431, 
432,433). While physicians may still order these 
procedures, they must provide written justi- 
fication for their use in order to be reimbursed 
by plans that adopt the medical necessity pro- 
posals. Most plans appear to be accepting these 
recommendations, and it has been estimated 
that full implementation may result in annual 
savings of as much as $200 million from limiting 
routine hospital admissions tests and as much as 
$100 million from curtailing the use of 68 surgi- 
cal and diagnostic procedures.?° 


Moreover, the national Blue Cross and Blue 
Shield Associations make over 100 coverage 
recommendations a year through the delibera- 
tions of their Medical Advisory Committee. 
With the advice of medical specialty groups, the 
Medical Advisory Committee issues suggested 
coverage guidelines. Usually these recommen- 
dations concern new technology and are ex- 
pressed in broad terms, but sometimes they 
specify indications for use. In addition, the com- 
mittee serves as a clearinghouse for information 
about the coverage policies of each of the Blue 
Cross and Blue Shield plans. 


The national Blue Cross and Blue Shield As- 
sociations consider only a limited number of 
coverage questions. Medical specialty group 


20In some areas where Blue Cross and Blue Shield plans are in- 
corporated together, they share the same medical advisory staff; in 
other areas, they operate as separate organizations with separate 
staffs. 
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representatives suggest which established proce- 
dures should be examined by the Medical Neces- 
sity Program, and the Medical Advisory Com- 
mittee’s agenda is largely determined by ques- 
tions raised by plans when they seek the nation- 
al associations advice. 


Cost and Efficacy 


Blue Cross and Blue Shield, like medicare, 
tend not to consider cost information directly in 
coverage deliberations. In a few cases, where a 
new procedure has been identified as a modifi- 
cation of an existing service which offers little 
additional benefit, however, payment for the 
new procedure has been fixed at the same level 
as that for the established technology. In addi- 
tion, cost has traditionally been one of the fac- 
tors examined in benefit package design. Re- 
cently, CEA was used in the development of a 
screening program when the National Cancer 
Institute contracted with the Blue Cross and 
Blue Shield Associations in 1976 to develop a 
model prepaid health service benefit package for 
cancer screening. Although the contents of this 
package originally were expected to be deter- 
mined by the consensus decisions of a panel of 
experts, the program that eventually was de- 
signed was heavily influenced by the results of a 
CEA of cancer screening submitted by one of the 
consultants hired for the project (168,434). 
Moreover, the creation of the Medical Necessity 
Program has introduced an approximation of 
cost-effectiveness methodology in Blue Cross 
and Blue Shield coverage decisions, because cost 
consciousness has encouraged coverage deter- 
minations that are more closely related to judg- 
ments made by the national medical specialty 
societies. 


Further, greater emphasis on efficacy data 
also has been promoted by Blue Cross and Blue 
Shield’s sponsorship of an Institute of Medicine 
study of the CT scanner and subsequent recom- 
mendation that coverage be restricted to uses 
that the Institute found to be supported by cur- 
rent evidence of efficacy. Yet, the extent to 
which these changes indicate that more cover- 
age decisions will be tied to efficacy information 
remains uncertain. Adequate evidence of the ef- 
ficacy and safety of medical technologies often 


is not available, and in the absence of such evi- 
dence, technologies judged to be nonexperi- 
mental and generally accepted usually receive 
positive coverage recommendations from both 
the plans and the national associations. 


Currently, however, Blue Cross and Blue 
Shield are studying contract and coding changes 
that ultimately may facilitate the application of 
cost and efficacy criteria. Although Blue Cross 
and Blue Shield have their own procedure code, 
plans are mandated to use it only when they 
deal with national account business (group ac- 
counts in which some individuals live beyond 
the boundaries of an individual plan); other use 
is discretionary and many local affiliates choose 
to use other codes for their general subscribers. 
At present, however, the national associations 
are attempting to revise and update the code, a 
move which may make it more attractive to 
member plans and more useful for the identifi- 
cation of questions concerning coverage issues 
or inappropriate use of a technology. More- 
over, the Blue Cross and Blue Shield Associa- 
tions are considering inserting new exclusionary 
language in their national account contract and 
recommending a model contract exclusion 
clause to its local affiliates in order to develop a 
firmer legal basis for coverage decision.”! In rec- 
ognition of this need, many plans have taken the 
initiative in recent years and developed more 


*1Thus, the Blue Cross Board of Directors and the Blue Shield 
Board of Directors will be asked to review and approve the follow- 
ing policy: 

To recommend to the Boards that each Plan be encouraged to re-ex- 
amine its contracts to assure adequate identification of medical neces- 
sity and experimental exclusions, and that the Boards offer the follow- 
ing language to each Plan for consideration. 

“Medically necessary services or supplies provided by hospital, 
physician or other provider to identify or treat an illness or injury and 
which, as determined by the Plan, are: 

1) Consistent with the symptom or diagnosis and treatment of the 
condition, disease, ailment or injury; 

2) Appropriate with regard to standards of good medical practice; 

3) Not primarily for the convenience of the member, his physician, or 
other provider; 

4) The most appropriate supply or level of service which can safely be 
provided to the member. When applied to an inpatient, it further 
means that the member's medical symptoms or condition require 
that the services or supplies cannot be safely provided to the mem- 
ber as an outpatient. 

Services and supplies which are experimental or investigational in 
nature; meaning any treatment, procedure, facility, equipment, drugs, 
drug usage, devices, or supplies not recognized as accepted medical 
practice or by the Plan and any of such items requiring federal or other 
governmental agency approval not granted at the time services were 
rendered.” 
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specific exclusionary language for their con- 
tracts. 


A few plans in California, Pennsylvania, 
Michigan, Massachusetts, and New York also 
appear to have taken the initiative in developing 
a somewhat more systematic coverage mecha- 
nism. Perhaps the most sophisticated decision- 
making process has been adopted by California 
Blue Shield, which addresses a series of ques- 
tions before making a coverage determination: 


1. Is the procedure experimental or investi- 

gational? 

Is it generally accepted? 

What is its relative safety? 

What does it cost? 

Is there a procedure which costs less and 

achieves the same result? 

Is it reproducible, i.e, can it be used by 

someone other than the original creator? 

7. Isit generally available? 

8. Does it make a difference in the manage- 
ment of patient outcomes or does it just 
serve an academic purpose? 


pe St 


sa 


Although stage of development and general ac- 
ceptance remain the most important criteria, 
California Blue Shield has succeeded in formally 


22For example, Washington, D.C., Blue Cross/Blue Shield 
issued the following statement in 1979 (59): 

Group Hospitalization, Inc., and Medical Services of D.C. have 
always been denied claims for hospital and medical services or supplies 
which are not medically necessary. We're now adding a contractual 
provision to all non-group contracts which specifically excludes serv- 
ices or supplies which are not, in the judgment of the corporation, med- 
ically necessary for the diagnosis or treatment of an illness. 


placing the concepts of cost and relative efficacy 
on its coverage agenda. In a few cases, where it 
has denied routine payment for a new procedure 
that it has determined costs more and achieves 
the same result as an existing technology, Cali- 
fornia Blue Shield has applied an approximation 
of CEA to its decisionmaking. Moreover, ulti- 
mate authority to make coverage determina- 
tions for the plan lies with the Medical Policy 
Committee, a 24-member group composed of 
physicians and 4 or 5 public representatives, 
which meets in a number of locations across the 
State and invites representatives of the local 
medical community and the appropriate special- 
ty societies to attend its sessions. At least 2 or 3 
weeks prior to the time the Medical Policy Com- 
mittee meets, the medical advisor distributes an 
agenda book containing all the information that 
has been collected about the procedures to be 
discussed. As a result, the Blue Shield coverage 
has achieved a high degree of acceptance within 
the California medical community, and the 
meetings of the Medical Advisory Committee 
often serve as a public forum for the generation 
of consensus about a procedure. Thus, even 
when a decision is made not to restrict payment 
for a technology, the public discussion of its 
merits may result in discouraging its use. Fur- 
ther, the high visibility of the Blue Shield cover- 
age process in California has augmented the 
State plan’s ability to identify coverage issues, 
because the developers of new procedures tend 
to seek the advice of the Medical Advisory 
Committee before requesting reimbursement 
(165,537). 


POTENTIAL USE OF CEA/CBA IN REIMBURSEMENT 
COVERAGE DECISIONS: GENERAL FINDINGS 


Cost-effectiveness analysis/cost-benefit anal- 
ysis (CEA/CBA) is based on the assumption 
that resources are limited and should be ration- 
ally allocated. Because it makes this assumption 
of limits and because it forces explicit considera- 
tion of alternatives for achieving goals, CEA/ 
CBA would appear to be able to contribute im- 
portant information: to decisionmakers con- 
cerned with acting as selective purchasers of 


health services. Yet its appeal for reimburse- 
ment officials seems severely tempered by the 
small number of well-conducted CEA/CBA 
studies available, methodological uncertainties 
of the techniques, and health policymakers’ lim- 
ited experience with their use. Moreover, the 
economic efficiency value embodied in cost-ef- 
fectiveness information may conflict with a 
number of other values prevalent in our health 
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care system, including: 1) the practitioner’s obli- 
gation to do the most for the patient, desire for 
relative certainty in making a diagnosis, and 
need to possess the freedom and flexibility to re- 
spond to the circumstances of the individual 
medical problem; 2) the patient's desire to re- 
ceive a full range of medical care, regardless of 
ability to pay; 3) society's desire to encourage 
innovation in order to ultimately improve care; 
and 4) society’s goals in terms of equity and 
other noneconomic values. 


On the other hand, the methodology for effi- 
cacy and safety analysis is more widely accepted 
than that for CEA.?3 Efficacy and safety studies 


3For a discussion of the conflict of medical economic efficiency 
with efficacy, see W.B. Schwartz and P.L. Joskow,’Medical Ef- 
ficacy Versus Economic Efficiency: A Conflict in Values,” N. Eng. 
J. Med., 1978 (548). 


form part of a relatively more established health 
policy tradition, and the value of protecting pa- 
tients from risky, unsafe, or unproven technol- 
ogy is less controversial (465) than weighing 
marginal cost and benefit tradeoffs and not re- 
imbursing for some potentially beneficial forms 
of care. For this reason, an approximation of the 
cost-effectiveness concept, in which cost-con- 
tainment goals encourage stricter adherence to 
safety and efficacy data might be more appro- 
priate and acceptable in reimbursement cover- 
age decisions than would formal CEA/CBA. 
One of the problems with this approach, how- 
ever, is that most technologies probably have 
some efficacy in some circumstances, and ra- 
tional allocation requires choosing among alter- 
native uses of technologies by considering mar- 
ginal costs and marginal benefits and relative ef- 
ficacy for a specific set of patient indications. 


COST EFFECTIVENESS IN RATESETTING 


In the last decade, the number of hospital 
ratesetting programs has increased rapidly. The 
central feature of this regulatory mechanism is 
the negotiation of hospital rates in advance of 
each operating year. Thus, Hellinger points out 
(294): 


The key difference between prospective rate 
setting and conventional methods of reimburse- 
ment is that hospitals are not paid the costs they 
naturally incur, nor are they free to unilaterally 
adjust their charges to cover their costs or their 
own interpretations of their financial require- 
ments; rather, they are paid at rates that are 
determined by another body and that are set in 
advance of, and considered fixed for, the pro- 
spective year. 


Various ratesetting mechanisms, however, 
differ considerably in their structure and opera- 
tions and in the methods and unit of payment 
they use to determine rates. For example, many 
programs have the power to set mandatory 
rates, whereas others rely on voluntary com- 
pliance. More than 20 programs are sponsored 
by Blue Cross, while 12 programs are funded by 
State governments, either through existing State 
agencies or newly formed independent commis- 
sions (154,155,294). 


Concern with the rapidly rising cost of health 
care has encouraged the growth of rate review 
mechanisms. In fact, Federal legislation has 
directly contributed to their development. Both 
the 1972 amendments to the Social Security Act 
(Public Law 92-603) and the 1974 National 
Health Planning and Resources Development 
Act (Public Law 93-641) provide for Federal 
funding of experiments in ratesetting. More- 
over, escalating medicaid expenditures have 
prompted an increasing number of States to 
consider establishing a rate review system. 


Basically, ratesetting agencies address the 
need to promote cost containment by encourag- 
ing hospitals to operate efficiently. By determin- 
ing in advance the amount of revenue that will 
be available, they reward hospitals that keep ac- 
tual costs below the established rates. Thus, 
Bauer points out (45): 


The advantages seemed obvious; if a hospital 
could know its payment rate before it rendered 
its services, it would have the highest possible 
motivation to see that these services were pro- 
duced in the most efficient manner, since its 
solvency would depend on keeping its spending 
within the limits of its anticipated revenues. The 


hospital would have positive incentives for effi- 
ciency as well, since if it could produce its serv- 
ice more cheaply than the predetermined rate 
had allowed, it could pocket the difference. 


In effect, then, the intent of rate review is to 
promote more efficient behavior by forcing hos- 
pitals to live within a fixed budget, identify and 
anticipate the costs of services and facilities, and 
make explicit decisions about the allocation of 
resources. 


The record of ratesetting agencies thus far, 
however, is inconclusive (45,293,294). One sig- 
nificant limitation is that the lack of adequate 
data, performance standards, and methodology 
makes it very difficult to set rates of payment 
that promote efficiency. As Bauer notes, ‘The 
central issue is how to set rates in a manner that 
will neither underpay nor overpay, but will en- 
courage each institution to increase the efficien- 
cy with which its services are provided” (45). 
Still another major problem is that ratesetting 
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programs tend to focus on hospital operating 
costs and do not concentrate on such other im- 
portant areas as the costs generated by physi- 
cians. 


Thus, it appears that use of CEA/CBA in rate 
review systems will have to await the develop- 
ment of more sophisticated ratesetting methods, 
more adequate data, and performance stand- 
ards, and closer liaison with other organiza- 
tions, such as health systems agencies. It may be 
that analysis concentrating on the net-cost end 
of the spectrum of analysis (e.g., cost per unit of 
service) is more appropriate for ratesetting. If 
so, ratesetting may be an area where increased 
numbers of CEA/CBA-like studies could be 
used. An interesting counterargument is that 
ratesetting could be based on the results of soci- 
ety-based CEA/CBAs and thereby would be less 
oriented to a narrower efficiency base and more 
toward social effectiveness. 
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6. 


Professional Standards 


Review Organizations 


INTRODUCTION AND BACKGROUND 


As noted in chapter 5, the Social Security Act 
authorizes the medicare program to pay only 
for services that are “reasonable and necessary,” 
and these criteria have been generally regarded 
as met when a new technology is perceived to 
have moved beyond experimental status toward 
full clinical application and acceptance by the 
local medical community. The current policy of 
the Department of Health and Human Services 
(DHHS) is to use “broad strokes to sketch the 
boundaries of accepted good medical practice, 
and (to leave) the fine-tuning of the system to 
the ‘back-end’ mechanism, the Professional 
Standards Review Organizations (PSROs)” 
(98). 


The PSRO program is one of the principal ex- 
pressions of Federal policy concerning the use of 
medical services. The PSRO program, estab- 
lished in 1972 by Public Law 92-603, is adminis- 
tered by the Health Standards and Quality Bu- 
reau (HSQB), formerly the Bureau of Quality 
Assurance, of the Health Care Financing Ad- 
ministration. The purposes of the program are 
to help improve the quality and control the costs 
of medical services reimbursed through Federal 
payment programs. The program operates by 
setting standards and criteria for the desired 
level and quality of medical services and by 
evaluating against these standards the services 
actually provided. This process is designed to 
ensure that payment will be made only when 
services are medically necessary. 


The PSRO program is based on the concept 
that medical professionals are the most appro- 
priate individuals to evaluate the quality of 
medical services and that effective peer review 
at the local level is the soundest method for en- 
suring the appropriate use of medical care re- 
sources and facilities. The PSRO program is 
made up of separate and independent organi- 


zations covering 195 geographic areas. Each 
PSRO must be substantially representative of all 
practicing physicians in an area. The PSRO pro- 
gram is new and is not yet fully implemented. 
Of the 203 PSRO areas in March 1977, only 120 
PSRO agencies had been funded; 100 were in 
“conditional” status; 20 were in “planning” 
status. By April 1979, the areas had been con- 
solidated to number 195. Of these, 182 had 
PSROs in “conditional” status, and 13 were be- 
ing planned or were inactive. 


Currently, PSROs are required to review in- 
stitutional care—care provided in hospitals and 
long-term care (nursing home) facilities—and 
are authorized to regulate ambulatory care. 
PSRO activities at present are concentrated on 
inpatient care provided in short-stay hospitals. 
The PSRO can delegate its review responsibil- 
ities to the hospitals, or, if a hospital is not 
qualified or not willing to conduct the reviews, 
the PSRO conducts them itself. Review of long- 
term care can be delegated if such care is a “dis- 
tinct part” of a short-stay hospital. 


PSROs conduct three types of review (42 
CFR, part 466): 


1. concurrent review of the medical necessity 
and appropriateness of admission to and 
continued stay in a hospital; 


2. medical care evaluation studies to assure 
the quality and improve the nature of the 
utilization of health care services; and 


3. analyses of health care practitioner, insti- 
tutional, and patient profiles. 


Medical care evaluation studies and profile 
analyses are used to focus concurrent review on 
problem areas. 
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As specified in the statute, PSROs review 
services to determine whether: 


e such services and items are or were medi- 
cally necessary; 

e the quality of such services meets profes- 
sionally recognized standards of health 
care; and 

@ such services and items proposed to be pro- 
vided in a hospital or other health care fa- 
cility on an inpatient basis could, consistent 
with the provision of appropriate medical 
care, be effectively provided on an outpa- 
tient basis or more economically in an in- 
patient health care facility of a different 


type. 


The law requires that PSROs use norms, cri- 
teria, and standards in evaluating medical serv- 
ices. This approach allows nonphysicians to 
perform many of the reviews and also enhances 
the objectivity of the review process. Standards 
are developed by a consensus of physicians, 
based on typical patterns of practice in the area 
and on such regional or national information as 


may be available and considered applicable by 
the PSRO. 


In its early stages, the PSRO program has 
concentrated on determining the need for hospi- 
talization. Now PSROs are beginning to move 
beyond the question of necessity for hospitaliza- 
tion to review of surgical procedures and review 
of ancillary services. 


PSRO decisions on medical care utilization 
and quality can be enforced in several ways. Re- 
imbursement for services provided can be with- 
held by medicare and medicaid (medicaid regu- 
lations are established in each State and vary 


REVIEW CRITERIA 


Criteria used in evaluating the quality of med- 
ical care are usually grouped according to struc- 
tural, process, and outcome measures. Struc- 
tural measures assess both the availability of 
medical care resources (e.g., facilities, equip- 
ment, and health care personnel) and the quali- 
tative aspects of medical care personnel (e.g., 
extent of educational background, specialty 


somewhat). For serious and repeated violations 
of PSRO standards, a physician’s right to be re- 
imbursed through medicare and medicaid can 
be suspended or revoked. 


Each State with three or more PSROs has a 
statewide Professional Standards Review Coun- 
cil. Among other duties, the statewide councils 
have the responsibility to disseminate informa- 
tion and data among the PSROs within the 
State. At the national level, a National Profes- 
sional Standards Review Council is established 
by law. This Council has several functions, one 
being to “provide for the development and dis- 
tribution, among Statewide Professional Stand- 
ards Review Councils and Professional Stand- 
ards Review Organizations of information and 
data which will assist such review councils and 
organizations in carrying out their duties and 
functions.” Such information is specified as in- 
cluding regional norms and standards. Local 
PSROs are not required to accept model stand- 
ards issued by the National Council. However, 
the National Council has authority to disap- 
prove local standards that deviate from model 
standards if the Council determines that the dif- 
ferences are not medically justified. The Nation- 
al PSRO Council has provided general guidance 
and sample criteria sets developed by several 
organizations, including the American Medical 
Association, under contract with DHHS. The 
main purpose of these contracts has been to de- 
velop criteria on medical necessity for hospitali- 
zation for different disease categories. HSQB 
hopes that technical assistance and norms and 
standards will have an important educational 
effect, as well as a direct effect on practice 
through reimbursement policy. 


board certification); process measures assess the 
appropriateness of the medical care that has 
been provided; and outcome measures reflect 
the effect of medical interventions on patient 
health status. PSROs, in their reviews, have re- 
lied primarily on process measures and, to a 
lesser extent, on outcome measures. 
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Generally, research on quality of care assur- 
ance and on the application of quality of medi- 
cal care techniques has been process /oriented, 
i.e., assessing the appropriateness of medical 
care as judged by the medical profession. PSRO 
concurrent review of admission to and con- 
tinued stay in hospitals is also process oriented. 
In practice, moreover, review activities of 
PSROs have been even more narrowly focused. 
The cost concerns of the Federal reimbursement 
program, together with the controversial nature 
and the uncertain state-of-the-art of reviewing 
the quality of medical care, have limited reviews 
to assessment of 1) whether a diagnosed prob- 
lem justified hospital admission, and 2) whether 
continued hospitalization was justified after a 
specified number of days of inpatient care. 


Recently, some PSROs have tried out slightly 
more sophisticated reviews than those based on 
length-of-stay indicators. Beginning in 1977, the 


Delmarva Foundation for Medical Care of the 
eastern shore of Maryland began to use a tech- 
nique called “care level and timeliness review” 
(CLTR) to help hospitals find and eliminate 
medically unnecessary hospital days (358). By 
retrospective audit, the Delmarva Foundation 
has evaluated both the timeliness of services and 
the level of medical care supplied. As alterna- 
tives to weekend and night admissions through 
the emergency room (which contribute to un- 
necessary hospitalization) and to lack of full 
laboratory services on weekends (which can 
lead to extra days of hospitalization until tests 
have been completed), the Foundation examined 
“swing beds” with additional staffing for obser- 
vation of patients to determine whether or not 
they should be admitted, and expansion of lab- 
oratory services from 6 to 7 days. In both cases, 
the reduction of unnecessary days was not 
enough to justify the extra costs. 


COST-EFFECTIVENESS APPLICATIONS 


In exploring the actual and potential appli- 
cability of cost-effective analysis/cost-benefit 
analysis (CEA/CBA) in the review activities of 
PSROs, it is necessary to delineate several sepa- 
rate concepts. First, PSROs do not normally 
take a specific medical problem or diagnosis and 
determine which of several effective treatment 
approaches is the least costly. Instead, they 
focus on reducing or eliminating unnecessary in- 
patient care. For a given diagnosis, PSROs per- 
form their review with reference to standards 
based on medical opinion that hospitalization 
for that diagnosis is or is not needed. They also 
review against standards of how many addition- 
al days of hospitalization are needed. The PSRO 
evaluation does not, systematically at least, ex- 
tend to a further determination that days of hos- 
pitalization beyond the standard (and the medi- 
cal care provided during this period) are not 
needed at all or could be just as effectively pro- 
vided through outpatient visits. With the explic- 
it goal of current PSRO reviews being to mini- 
mize unnecessary hospitalization—not to deter- 
mine whether hospitalization, outpatient care, 
or no medical care at all is most effective for a 


given medical problem—PSROs are focused on 
reducing medical care that is of almost no value 
to the patient, but for which reimbursement is 
nevertheless being provided. 


Second, although outcome measures, or the 
effect of medical interventions on patient health 
status, are often viewed as the best criteria for 
evaluating the quality of medical care, these 
have been the most difficult criteria to develop. 
Hence the reliance on process, or medical opin- 
ion, criteria. For most medical problems, how- 
ever, alternative avenues of treatment—not 
clear-cut, single choices—have been the rule. 
Furthermore, the effectiveness of even estab- 
lished treatments is continually under question. 
Taken together, the use of several methods of 
treatment for specific medical problems and the 
often uncertain correlations between specific 
medical interventions and the patient’s health 
status make the development of additional crite- 
ria to decide which interventions should or 
should not be reimbursed very difficult. Never- 
theless, the development of such criteria, es- 
pecially for many of the common interventions 
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that are subject to review by PSROs, is not im- 
possible. Because PSROs in effect concentrate 
on deviant practice by establishing ranges of ac- 
ceptable behavior, boundaries can be set to al- 
low reasonable divergence based on the possi- 
bilities of individual patient response. In con- 
cept at least, the ranges of acceptable behavior 
by institutions or physicians under review could 
be based on information that incorporates cost 
or cost-effectiveness-related data. The ranges 
could also be based on a more, though only 
slightly, common form of data—efficacy and 
safety. 


These two aspects of the use of medical tech- 
nologies—efficacy and safety, and cost effec- 
tiveness—together define the appropriate use of 
technologies. The PSRO legislation established 
a framework by which appropriate use of medi- 
cal technologies could be evaluated by physi- 
cians acting in organized groups rather than as 
individuals. PSRO’s decisions, however, are 
still based largely on traditional sources of in- 
formation, so customary practice patterns, 
whether appropriate or not, become accepted as 
standard. The lack of scientifically derived in- 
formation on indications for use and, especially, 
on the cost effectiveness of various technologi- 
cal applications hampers the development of ap- 
propriate standards. Provided with such infor- 
mation, PSROs could perhaps become a more 
effective mechanism for evaluating medical 
care. In its absence, PSROs are developing local 
standards for medical services based primarily 
on prevailing patterns of medical practice, with 
little comparison of costs to benefits. 


On the other hand, cost-effectiveness ap- 
proaches have been used in managing some 
PSROs and in evaluations of the national PSRO 
program itself. Even with the current focus on 
reducing medically unnecessary inpatient days 
in short-stay hospitals, cost-effectiveness tech- 
niques can be used for PSRO management pur- 
poses. For example, a PSRO might use analyti- 
cal techniques to decide which of a number of 
diagnoses to focus on to reduce, by a targeted 
number, the medically unnecessary days in a 
given year. 


Incentives to use _ cost-effectiveness ap- 
proaches to reduce medically unnecessary bed 


days already exist and have influenced specific 
PSROs in choosing among problem areas. The 
PSROs’ administrative budgets are funded by 
annual Federal grants, and their hospital review 
activities for medicare, medicaid, and the ma- 
ternal and child health and crippled children 
programs’ patients are paid from the medicare 
trust funds (Public Law 94-182). Congress set a 
limit on the use of fiscal year 1979 medicare 
trust funds for such hospital reviews (44 F.R. 
26770, May 7, 1979). 


Each PSRO is evaluated annually with regard 
to its impact on reducing medically unnecessary 
hospitalization and must negotiate formal annu- 
al “objectives” with the Federal Government. 
Objectives are stated in terms of “impact” (e.g., 
“reduce the incidence of medically unnecessary 
days in PSRO hospitals to 5 percent or less as 
measured by CLTR”). 


Two Maryland PSROs, the Delmarva Foun- 
dation for Medical Care and the Baltimore City 
PSRO, illustrate the effect of budget constraints 
on seeking alternative approaches to reducing 
unnecessary hospitalization (358). In late 1977, 
the Delmarva Foundation for Medical Care de- 
cided that its own collection of hospital data did 
not enhance its capability to reduce unnecessary 
hospitalization or assist materially in iden- 
tifying problem areas. By deciding to eliminate 
data collection as a PSRO activity and accepting 
discharge abstracts prepared by the hospitals, 
this PSRO was able to reduce the concurrent re- 
view cost per discharge from $6.50 to $5.00. 


The Delmarva Foundation also changed its 
concurrent review process in order to increase 
its efficiency. Nonessential tasks such as coding, 
norm assignment, and abstracting are no longer 
performed by reviewers. Review procedures 
have been simplified to concentrate only on the 
clinical indications for the need for continued 
hospitalization. A patient receiving intravenous 
therapy, for example, is automatically consid- 
ered to be appropriately placed in the hospital. 
By focusing only every other day on the clinical 
indications for hospitalization, as opposed to 
assigning a normative length of stay in advance 
for a particular diagnosis, this PSRO dramati- 
cally shortened the amount of time required for 
review per patient. As a result, the concurrent 


review cost per discharge dropped from $5.00 to 
$3.25, yet the PSRO’s capacity to reduce unnec- 
essary hospitalization, as measured by CLTR, 
has not been affected. OTA has no information 
on whether the quality of the data itself was af- 
fected. 


In 1979, in an attempt to cut costs further, 
Delmarva began exempting hospitals with low 
levels of inappropriate use from admission re- 
view, continued stay review, or total review. 
Consequently, concurrent review costs have 
dropped below $2.50 per discharge. The impact 
of this action on appropriate hospitalization has 
not yet been evaluated. 


Recently, the Delmarva Foundation has at- 
tempted to improve its utilization review proc- 
ess by employing, on a test basis, the appropri- 
ateness evaluation protocol (AEP), a recently 
developed technique that compares care deliv- 
ered with an objective ideal. Like CLTR, AEP 
measures unnecessary days of care created by 
organizational problems and clinical judgments 
that are grossly out of tune with mainstream 
practices. AEP goes one step beyond CLTR, 
however, because it questions clinical practices 
that are in local general use. In addition to pro- 
viding more timely and complete information 
than CLTR, AEP promises to be less costly. 


This same PSRO has begun using cost-effec- 
tiveness-type analyses to identify unnecessary 
days of care and to change modes of physician 
practice on a diagosis-specific basis. In selecting 
topics for evaluation, Delmarva focuses on find- 
ing relatively low-cost opportunities for making 
substantial impacts. A model developed by the 
PSRO’s statistician is used to select topics for 
regional medical care evaluations that will have 
the most impact. This model not only identifies 
differences in utilization between the PSRO area 
and regional norms, but also identifies dif- 
ferences within the PSRO area in utilization that 
can be changed through local educational pro- 
grams. 


The Baltimore City PSRO, partially in re- 
sponse to the cap on review costs, has taken a 
different approach to maximizing its impact 
under financial constraints. Specifically, it has 
used its data base and computer capabilities to 
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rank hospital and diagnoses according to excess 
utilization rates, thereby enabling the PSRO to 
focus its attention on those hospitals and diag- 
noses that produce the highest incidence of un- 
necessary hospital days. 


Excessive hospital days may not be true in- 
dicators of excessive costs. Nevertheless, it has 
been the experience of the Baltimore PSRO that 
hospitals which have problems with diagnosis- 
specific lengths-of-stay also tend to have prob- 
lems with high costs and higher than average 
death rates. Given these facts and the PSRO’s 
limited budget, the Baltimore organization be- 
lieves that length-of-stay analysis is an appro- 
priate way to identify inappropriate and unnec- 
essary medical care. By identifying problem 
areas under the length-of-stay analyses, the 
PSRO is able to focus its concurrent review, 
medical care evaluation, and profile analysis ef- 
forts on those hospitals and diagnoses where the 
impact can be greatest. 


This technique seems to have worked for the 
Baltimore PSRO. In the first study of this type, | 
that PSRO undertook a very careful review of 
the hospital that ranked first in excess utiliza- 
tion. Review of the hospital’s data indicated 
potentially serious problems with several spe- 
cific diagnostic and cost levels. The hospital has 
since confirmed the existence of these problems 
and corrected the excess utilization problem. 


Evaluations of the national PSRO program 
have been conducted by the General Accounting 
Office (GAO) (243), the Congressional Budget 
Office (CBO) (121), and DHHS (285,288). Cur- 
rently, GAO is conducting a general review of 
PSROs, with the intent of focusing future 
studies on more specific components of the pro- 
gram (404). CBO is also currently updating its 
evaluation (122), as is DHHS (138). 


The GAO report was a general review of the 
progress and problems in establishing PSROs. 
The CBO report more specifically addressed the 
effect of PSROs on health care costs and wheth- 
er the estimated savings (if any) derived from 
the program outweighed the program costs. 


DHHS's 1977 study (288) found no reduction 
in unnecessary hospital days was achieved 
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through PSRO activities. Its 1978 medicare rate 
study (285) concluded that there was a 1.5-per- 
cent utilization reduction, leading to an esti- 
mated savings of $50.5 million. With medicare 
review costs of $45.9 million, the study con- 
cluded, there was a net benefit of $4.6 million (a 
savings-to-cost ratio of 1.1 to 1) or 10 percent of 
review costs. 


CBO analyzed the same data used by DHHS, 
but reached a different conclusion. CBO ad- 
dressed the effectiveness of PSROs by posing the 
following three questions: 


1. How effective is the program in reducing 
hospital utilization? 

2. Are the savings associated with the pro- 
gram large enough to justify the costs of 
the program itself? 

3. Are the program’s net-savings large 
enough to warrant the expectation that 
PSROs will play a major role in contain- 
ing health care costs? 


CBO concluded that PSROs have brought 
about a decrease in days of care of roughly 2 
percent for medicare beneficiaries in short-stay 
hospitals. Its analysis of the DHHS data led 
CBO to the conclusion that utilization savings 
would be about 30 percent less than review 
costs, in contrast to DHHS’s conclusion that 
savings would be 10 percent greater then review 
costs. (Both estimates, however, were subject to 
wide margins of error.) CBO further concluded 


that, even if the 10-percent net savings were ac- 
cepted, those savings would still be extremely 
small relative to Federal expenditures for acute 
patient care—amounting to less than 0.1 percent 
of medicare reimbursements for inpatient care 
in short-stay general hospitals. 


One problematic issue is the opposite conclu- 
sions reached by CBO and DHHS after analyz- 
ing the same data, i.e., utilization savings 30 
percent less than review costs versus savings 10 
percent greater than review costs. The CBO 
report lists a number of methodological prob- 
lems that could shift the conclusion of the cost 
effectiveness of the PSRO program either to a 
net loss or to a net savings position, and the per- 
suasiveness of either conclusion might well rest 
in the eye of the beholder. Perhaps more impor- 
tantly, however, these analyses do not examine 
costs in relation to changes in health outcomes 
that may result from PSRO reviews. Interesting- 
ly, these evaluations, which sometimes show a 
small cost savings and sometimes show a small 
net cost increase, are critiqued in terms of the 
value of the national PSRO effect. Implicit in 
such a criterion is a view of PSROs as a cost- 
containment mechanism. Yet, if this is indeed 
the rationale of the PSRO program, what does it 
matter if the program does cost slightly less than 
the amount it saves? That net saving is still in- 
finitesimal compared to the total cost of the pro- 
grams that the PSROs are supposed to be con- 
straining. 


USE OF CEA/CBA BY PSROs: GENERAL FINDINGS 


The PSRO program has the broad responsi- 
bility of seeing that Federal funds are used for 
health care services and items that are medically 
necessary, meet professionally recognized 
standards of care, and are provided at the most 
economical level possible consistent with quali- 
ty care. In practice, however, the program has 
concentrated on weeding out wasteful care. 
Even in this limited approach, cost-effectiveness 
approaches (loosely, and not formally, defined) 
have potential applicability in: 1) setting stand- 
ards of medical care against which actual prac- 
tices are judged, 2) the internal management of 


individual PSROs, and 3) evaluations of the na- 
tional PSRO effort. 


It is theoretically possible that standards 
based on CEA/CBA or on other analyses incor- 
porating consideration of costs in relation to ef- 
fectiveness could be developed at the national or 
regional level and adapted for local use by 
PSROs. Development of such criteria will be 
difficult, and acceptance by PSROs may not be 
gained easily. Criteria for identifying quality 
medical care have gone beyond gross measures 
such as length-of-stay norms to (still modest) 


criteria such as the timeliness of services and the 
level of medical care supplied (i.e., CLTR) and 
objective standards such as the AEP. If usable 
and relevant data on the appropriate uses of 
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various medical technologies are developed and 
made available to PSROs, there may be con- 
siderable potential for applying CEA/CBA in 
PSRO review activities. 
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Health Planning Programs 


INTRODUCTION AND BACKGROUND 


Some of the earliest known uses of cost-bene- 
fit analysis (CBA) were associated with health 
planning. In the middle of the 17th century, for 
instance, a prominent English physician, Rich- 
ard Petty, favored society’s investing more in 
medicine because the value of saved human life 
far exceeded its cost (194). In this country, a 
similar argument was made by Lemuel Shat- 
tuck, who in his famous 1850 report used CBA 
to justify his proposal for sanitary reforms in 
Boston (559). It is interesting to note that most, 
if not all, of the early applications of cost-effec- 
tiveness analysis/cost-benefit analysis (CEA/ 
CBA) to health care were used by the health 
profession itself to justify further investment in 
the public’s health. Unlike the main proponents 
of CEA/CBA today—economists and others 
who argue that analytical techniques are needed 
to help society spend its health care dollars more 
efficiently—earlier proponents and users of 
CEA/CBA were health professionals who ar- 
gued that society should spend more money on 
its health care. These professionals rested their 
argument on a concept that still dominates the 
discipline of health planning today—the concept 
of “medical need.” . 


“Medical need” is not well-defined. In the dis- 
cussion that follows, the term “medical need” is 
used to refer to that health benefit which an in- 
dividual or a population could and “should” re- 
ceive from a given health intervention or a pre- 
vention program. Thus understood, medical 
need is subject to change, depending on the state 
of medical and health knowledge and the avail- 
ability of resources, and it also carries with it an 
element of equity. Thus, whereas a heart trans- 
plant was not “needed” 20 years ago because the 
procedure was not possible at that time, the de- 
termination as to whether there is a “need” 
today will depend on the safety and efficacy of 
the procedure (i.e., the state of knowledge), the 
personnel, equipment, and money available for 


delivering it, and a notion of whether it 
“should” be available. This last consideration— 
should—is difficult to define because society’s 
underlying values are so intertwined with it. A 
95-year-old man, for example, might not qual- 
ify for a publicly funded heart transplant under 
any circumstances. In general, then, the concept 
of “medical need” is defined as some sort of ab- 
solute medical requirement, within some 
bounds of reasonableness. This concept is in- 
consistent with generally accepted economic 
principles which are based on relative values 
such as price, utility, and alternative uses of re- 
sources. 


Over the years, whether the issue has been 
manpower (353), public health departments 
(175), hospital construction, or comprehensive 
health planning, the principal consideration in 
planning has been “need:” If the planner could 
show a medical need for the resource or the 
service, the objective was set, the question of 
financial feasibility often being a secondary con- 
sideration. 


When funds had to be specifically appropri- 
ated to meet an identified ‘“‘need,’’ however, an 
implicit cost-benefit tradeoff must have been 
made. For instance, the 1933 Lee-Jones Report 
(353) calling for more physicians would have re- 
quired additional training funding; the 1945 
Emerson Report (175) calling for complete pub- 
lic health coverage required additional Federal 
funding; and the 1946 Hill-Burton Act calling 
for increased hospital construction also required 
Federal funding. Although there is no evidence 
that formal CEA/CBAs were done in any of 
these cases, funding decisions were made in a 
political arena which implicitly weighed benefits 
against costs: When it appeared that benefits 
outweighed costs, funds were more likely to be 
appropriated; when the reverse seemed true, 
funding was curtailed. In the case of traditional 
public health measures, the Federal Government 


i 
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increased funding from the mid-1930’'s until the 
early 1950's, at which point it perceived that ad- 
ditional costs (investment) would exceed addi- 
tional benefits and therefore it curtailed funding 
even though the original goals had never been 
met (532). Recently, Federal funding for hospi- 
tal construction has been similarly curtailed, as 
has funding for physician training. Although 
the essential ingredients for a CEA/CBA were 
present, formal economic analyses of these pro- 
grams were not ordinarily performed. 


Most health care resource decisions today are 
made in an economic semivacuum. Owing in 
large part to the third-party reimbursement sys- 
tem and to the relative ease with which the 


health care system can obtain private capital, 
the recovery of investment has been all but 
guaranteed. In other words, the cost-benefit 
tradeoff has been distorted. Many believe that 
this distortion has led to the situation in which 
health care costs have escalated disproportion- 
ately to benefits, a situation that began with the 
advent of private health insurance in the 1930's 
and 1940's and became even more acute with the 
1965 medicare and medicaid amendments to the 
Social Security Act. Adding to the problem has 
been the largely unrestrained diffusion of tech- 
nological achievements of our time. Basically, 
the unwritten rule has been that if a technolog- 
ical advancement has medical merit it should be 
adopted. 


THE LAW AND CONGRESSIONAL INTENT 


Recent legislative measures reflect public con- 
cern over the rising costs of health care. Partly 
in anticipation of the economic effects of the 
medicare and medicaid amendments and also to 
help rationalize the health care system, in 1966 
Congress passed both the comprehensive health 
planning (CHP) amendments and the regional 
medical program (RMP). By that time, scholars 
including Roemer had already noted a positive 
correlation between unneeded beds and unnec- 
essary hospitalization (521), and other research- 
ers were beginning to suggest that some health 
care procedures, especially surgery, were being 
performed unnecessarily. An underlying prem- 
ise of the CHP/RMP laws was a belief that the 
pluralistic and fragmented health care system 
simply needed a catalyst to help its elements co- 
operate with one another. Cooperation among 
these elements, it was believed, would reduce 
duplication of services and facilities, and future 
decisions would therefore be “cost effective.” 
Planning and compliance under the CHP/RMP 
laws were strictly voluntary. These laws envi- 
sioned a health planning system based on 
“need:” If a particular service or facility was 
“needed,” then it must be worth the cost. 


As it became apparent that voluntary plan- 
ning efforts were not effectively constraining the 
health care system, Congress and individual 


States began searching for stronger measures to 
contain costs. First, in 1964, New York State en- 
acted a certificate-of-need (CON) law that em- 
powered its planning agencies to deny reim- 
bursement to hospitals for large capital expendi- 
tures unless the agency found that there was a 
“need” for the service to be provided. Subse- 
quently, several other States enacted similar 
laws. These State laws were strengthened by 
Congress in 1972 when it passed section 1122 of 
the Social Security Act, allowing Federal funds 
for capital expenditures to be withheld if large 
capital projects were not approved by State 
planning agencies. 


In 1974, Congress enacted the National 
Health Planning and Resource Development Act 
(Public Law 93-641), which required all States 
to legislate CON laws. The main intent of this 
Act was to ensure that health resource decisions 
were based on cost as well as need. Section 1513 
of the Act demonstrates concern for health sta- 
tus, cost, and accessibility, and sections 1502 
(national health priorities) and 1532 (CON) 
show similar concerns. 


By 1979, Congress had passed amendments to 
the Act specifying “cost effectiveness” as one of 
the criteria that local health planning agencies— 
health systems agencies (HSAs)—must use in 
their review of the appropriateness of a health 
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service. Specifically, 
amended to read: 


section 1513(g) was 


... In making the appropriateness review 

. . Of a health service, each health systems 

agency shall at least consider the need for the 

service, its accessibility and availability, finan- 

cial viability, cost effectiveness (italics added), 
and the quality of service provided. 


Consideration of “cost effectiveness” was also 
required of the State planning agencies—State 
health planning and development agencies 
(SHPDAs)—in their review of appropriateness. 


The use of the term “cost effectiveness” in the 
1979 amendments suggests a growing emphasis 
on a cost-benefit type of tradeoff. There is no 
evidence that Congress intended that HSAs and 
SHPDAs do anything as elaborate as formal 
CEA/CBAs, but there is evidence that Congress 
wanted these planning agencies to consider dur- 
ing the planning process both the effectiveness 
of the health care system and its cost. The 1979 
amendment to section 1502 (national health pri- 
orities), for example, specified the following as 
national priorities: 


The adoption of policies which will (A) con- 
tain the rapidly rising costs of health care deliv- 
ery, (B) insure more appropriate use of health 
care systems, and (C) promote greater efficiency 
in the health care delivery systems . . . and the 
development and use of cost saving technol- 


ory), 
and 


The strengthening of competitive forces in the 
health care services industry wherever competi- 
tion and consumer choice can constructively 
serve .. . to advance the purposes of quality as- 
surance, cost-effectiveness, and access. 


In addition, other parts of the 1979 amendments 
severely restrict HSAs’ authority over health 
maintenance organizations (HMOs), in the ap- 
parent belief that HMOs have inherent cost-ef- 
fective incentives, an issue more fully consid- 
ered in chapter 10 of this volume. 


There is evidence, then, that Congress intends 
that the Department of Health and Human Serv- 
ices (DHHS) consider both effects and costs in 
its health care resource considerations and that 
this intent has become more explicit over the 
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years. The National Health Planning and Re- 
source Development Act and amendments have 
provided five vehicles through which health re- 
source allocation decisions can, theoretically, be 
affected: 


1. CON for large capital expenditures or ad- 
ditions of new services, 

2. Federal funds review for certain health 
programs, 

3. appropriateness review for existing facili- 
ties, 

4. area health services development fund for 
planning grants for “needed” services, and 

5. closure and conversion funding (title XVI) 
for assisting facilities to close “unneeded” 
facilities. 


Of the five, CON has been the main vehicle 
available to planning agencies for affecting re- 
source allocation. Appropriateness review car- 
ries with it no sanctions,! and neither the area 
health services development fund nor closure 
and conversion funding (title XVI) has been 
funded to date (189). 


The 1974 National Health Planning and Re- 
source Development Act contained several pro- 
visions intended to ensure that planning agen- 
cies would have technical and analytical capa- 
bilities as well as assistance presumably for 
analyses such as CEA/CBA. First, the 1974 law 
authorized funding the HSAs at a significantly 
higher level than the old CHP agencies and also 
specified that HSA staff are to have expertise in 
administration, in the gathering and analysis of 
data, in health planning, in financial and eco- 
nomic analysis, and in the development and use 
of health resources. Second, the law provided 
for the use of consultants. Third, it provided for 
Regional Centers for Health Planning to provide 
technical assistance to agencies. Fourth, it pro- 
vided for a National Health Planning Informa- 
tion Center. 


The law makes it clear, however, that HSAs 
are not supposed to gather data, a restriction 
which is significant because available data are 
not sufficient to perform most CEA/CBA-type 
studies. In addition, it should be noted that HSA 
funding has been appropriated at a significantly 


‘Federal Register, Dec. 11, 1979. 
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HSAs have only 90 days for the entire CON 
process, another factor that may limit HSAs 
performance of CEA/CBA studies is time. 


lower level than was authorized, which in itself 
could preclude HSAs’ ability to perform valid 


studies. Finally, it should be noted that since 


REGULATIONS, GUIDELINES, AND ASSISTANCE 


Like Congress, DHHS is explicit in its desire 
that health care resource costs be balanced 
against benefits. In 1979, the Secretary issued 
planning guidelines to State agencies 
(SHPDAs), State Health Coordinating Councils 
(SHCCs), and local planning agencies (HSAs) 
(287). Contained in these guidelines was a 
message to planning agencies that their assess- 
ments of the costs of implementing goals and 
objectives should be made in context with the 
expected effects of such implementation both on 
the health status of the area residents and on the 
health care system and its costs. 


This message is a clear departure from previ- 
ous ones, those messages, in effect, being that 
planning should be based strictly on “need” first 
and “efficiency” second. The departure from 
previous policy does not neglect the concept of 
need in the planning process; indeed, identifying 
and planning for the health needs of the local 
population is still an important theme in the 
new guidelines. What distinguishes the new 
guidelines is that cost containment is apparently 
as dominant a theme as need (287). 


Cost containment is one of the fundmental 
purposes of the health planning program. Sec- 
tion 1513(b)(3) of Public Law 93-64 requires 
comparison of cost and benefits. 


Previously, the concept of cost containment was 
generally expressed in terms of eliminating “un- 
necessary” costs, referring to those costs that 
produce little or no benefit. The nuances of the 
latest guidelines suggest that other costs are not 
justified because they produce insufficient bene- 
fits. 


To make judgments concerning the adequacy 
of benefits in relation to costs, agencies must 
first be able to identify changes in health status 
indicators that result from health care invest- 
ments. The Secretary states (287): 


The HSA should describe in its HSP (Health 
Systems Plan) the expected effects of health sys- 
tems goals upon the improvement in the popula- 
tion’s health status . . . as measured by “Indica- 
tors” (which) are quantifiable measures chosen 
to reflect the health status of the population or 
to represent how well the health system is per- 
forming. Direct indicators (such as infant mor- 
tality or disability days, and cost per patient 
day, or time, or distance from primary care) 
measure the level and/or change in community 
health and in health systems performance. Indi- 
rect indicators (such as percent of the area popu- 
lation with income below the poverty level or 
the number of houses lacking adequate plumb- 
ing) indicate social or environmental conditions 
which have been attributed to affecting the 
health of the area’s residents. 


Then, goals are assessed in terms of expected 


payoffs (287): 


The development of health systems goals is 
based on an assessment of the health service 
needs of the present and projected population, 
and on an assessment of health services in terms 
of availability, accessibility, cost, continuity, 
acceptability, and quality. The HSP summarizes 
the HSA’s assessment of health service alterna- 
tives. The summary(ies) should focus on possi- 
ble tradeoffs between characteristics, such as ac- 
cessibility and cost, which will affect the selec- 
tion of goals, and priorities among selected 
goals. 


The HSP summarizes the major alternative ac- 
tions considered for accomplishing each objec- 
tive. The choice between alternative actions 
should obviously be predicated on technical, 
political, and administrative feasibility, as well 
as cost effectiveness analysis. 


In discussing the importance of cost, the Sec- 


retary states (287): 


The HSA must consider costs when it sets 
goals and objectives in the HSP, as well as when 
it develops the recommended actions in both the 
HSP and AIP (annual implementation plan). 


Ch. 7—Health Planning Programs ® 75 


Costs of goals and objectives also should be de- 
scribed to the extent feasible. Such costs will be 
derived from detailed estimates of the costs of 
recommended actions in the HSP and AIP. 


In addition to analyzing the costs of potential 
goals, objectives, or actions, the HSA should 
also analyze and compare their expected bene- 
fits. The expected effect of any objective or ac- 
tion upon the area’s health status and its eco- 
nomic effects as well as any anticipated im- 
provement in health system performance should 
be specified. The benefits of each alternative 
should be compared to the costs of that alterna- 
tive. 


Furthermore, the Secretary specifically refers 
HSAs to the results of a Health Resources Ad- 
ministration (HRA) contract product which pro- 
vides the conceptual and technical framework 
for a formal CBA in health planning (287).? 


HRA has funded numerous other contracts to 
assist in developing a framework for resource 
allocation. At least one is a formal CEA/CBA- 
type analysis (68), but most are more financially 
oriented, providing guidelines on capital budg- 


?The paper, “Consideration of Benefits and Costs: A Conceptual 
Framework,” was written by D. A. Dittman and K. R. Smith 
(146). In it, the authors provide guidelines by which planners can 
set priorities, choose between them, and perform formal cost- 
benefit analyses. 


eting (particularly lifecycle cost analysis, which 
includes not only direct capital financing but 
also future operating expenditures) (11,363, 
424). In February 1980, HRA, through its Bu- 
reau of Health Facilities, sponsored a conference 
on health care capital in which one of the prin- 
cipal topics was the relationship between capital 
and operating costs (286). It is generally con- 
ceded that such comprehensive costing is impor- 
tant for CEA/CBA-type studies. Also important 
is estimating health impacts, and for that, HRA 
has funded the development of methodologies 
such as the health status impact study by Kisch, 
et al. (315). 


HRA appears to be making a deliberate and 
orderly attempt to introduce, interpret, and ex- 
pand the concept of the balancing of costs and 
benefits in health planning. At the same time, 
there appears to be a justified realization at 
HRA that the state-of-the-art of CEA/CBA may 
be too immature for formal, routine use of these 
analytic techniques. HRA’s emphasis at present, 
therefore, is on organizational development at 
both the State and local agency level, on “need” 
assessment, and on HSP development (211). 
Meanwhile, the more sophisticated elements of 
CEA/CBA methodology (e.g., lifecycle capital 
budgeting and health status assessment) are be- 
ing studied and slowly introduced and dissemi- 
nated to the agency level. 


CURRENT USE OF ECONOMIC ANALYSIS 


The cautious approach that HRA has adopted 
with respect to the use of CEA/CBA-type tech- 
niques for planning is reflected in an analysis of 
HSA activities. An OTA-funded study found no 
evidence that HSAs have used formal CEA/ 
CBA to assist in making resource allocation 
decisions (358).3 In this study, HSAs most likely 
to employ formal economic analyses during the 
planning and decisionmaking process were iden- 
tified through an intensive interview process. 


3This study, conducted for OTA by Lewin and Associates, ex- 
amined the use of CEA/CBA by HSAs and by PSROs. 


Cost-Benefit Analysis 


OTA found one example of the use of CBA by 
an HSA (Miami Valley HSA, Dayton, Ohio) for 
the majority of its recommendations in the AIP. 
For the 1980 plan, the Miami Valley HSA in- 
cluded 54 CBAs. In general, the planning agen- 
cy’s staff contrasted the costs and cost savings of 
a given health program with the productive val- 
ue of the estimated years of life saved by that 
program, thus valuing life in terms of lost wages 
averted (i.e., the human capital method of valu- 
ing life*). The HSA’s analyses were presented as 
~ “This method of valuing life is discussed in ch. 3 of Background 


Paper #1: Methodological Issues and Literature Review, prepared 
by OTA in conjunction with this assessment. 
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supporting evidence for actions that had already 
been taken and objectives that already had been 
set. The methods of the analyses were simple 
and did not require large amounts of staff time. 
The HSA staff estimated that between 2 and 10 
person-days were spent on each analysis, and 
indicated that most analyses fell at the low end 
of this range. 


Although the HSA’s analysts had reviewed 
the pertinent literature for appropriate parame- 
ters for the analyses, in most cases the judgment 
applied to these parameters was solely that of 
the analysts. For example, a local health educa- 
tion organization had estimated that a health 
education program could save family medical 
bills up to $290 per year; since the HSA analysts 
intuitively felt that a savings of $290 per family 
per year was too high and could not be corrobo- 
rated, they arbitrarily chose to use in the analy- 
sis a savings of only $50, which they discounted 
at 2.5 percent over a 15-year period. Even with 
this major modification of benefit structure, the 
benefit/cost ratio in the analysis was calculated 
to be 26:1. On the strength of this analysis, the 
Miami Valley HSA included in the AIP a recom- 
mendation to expand the health education pro- 
gram to include 6,000 low-income families. 


The supporting information for other similar 
analyses in the AIP section of the HSP was very 
sparse, too. Although elsewhere in the plan 
document there was a general introduction to 
CEA and CBA concepts, nowhere in the presen- 
tation of the analyses was there a discussion of 
the assumptions required by each of the anal- 
yses or the levels of uncertainty associated with 
them. It was clear that the HSA staff had neither 
the time, the resources, nor the expertise to 
carry out valid CBA studies. (Indeed, for many 
of the programs which they evaluated, sufficient 
efficacy information is probably not available 
for anyone to perform good studies). Yet, the 
studies that were done have been incorporated 
into the AIP and apparently have gone virtually 
unchallenged. 


This example is indicative of the bind in 
which the planning agencies find themselves. 
They are encouraged, and even mandated, to do 
more than they are perhaps capable of doing. 
The skills, data, and funds that the agencies 


need to perform high-quality CEA/CBA-type 
studies are not available. It is important to em- 
phasize here that OTA found no indication that 
the Miami Valley HSA or any other has used or 
intends to use CBA in its resource allocation 
decisionmaking process. CON deliberations and 
other resource allocation decisions are not sub- 


jected to CBA. 


Financial Impact Analysis 


Although, for resource allocation decisions, 
OTA found no evidence of the use of CEA/CBA 
by HSAs (in the sense that health effects were 
estimated and compared to costs), it did find 
evidence of the use of related analyses. These 
“financial impact analyses” or “net cost studies” 
can take one of several forms, some of which 
are broader than others. 


One financial impact analysis, for example, 
addressed the impact of terminating/consolidat- 
ing a hospital service. That analysis addressed 
the following issues: 


e the cost savings from closure or consolidation 
of health care services; 

e the financial impact on the hospital's termi- 
nating/losing the service and the possibility 
that the hospital’s rates would have to be in- 
creased to replace “needed” revenue; 

e the possibility that an offsetting service could 
be found to replace lost revenue; 

e the impact on established physician practice 
patterns; 

e the impact on existing physical plants; and 

e the possibility that other facilities could be 
used for “needed” services which will no 
longer be provided by the facility facing the 
closure. 


Note that this analysis, a typical financial im- 
pact analysis, does not consider changes in 
health status, although it does consider both 
public and private costs (a point which is dis- 
cussed more fully later), and is used to measure 
changes in accessibility to services. 


Another example of a financial impact analy- 
sis was an analysis an HMO submitted with its 
application for a CON for an open heart surgery 
unit. That analysis demonstrated that the HMO 
would save money by building and operating its 
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own unit rather than contracting for the service 
as it was then doing. In its review of the HMO’s 
application, the HSA countered with its own 
study showing that net societal costs would be 
increased by the addition of the HMO’s unit, be- 
cause there would be excessive capacity in the 
area and existing facilities would lose business. 
Again, this finding may represent the clash of 
private versus public costs and benefits. | 


Most HSAs undertake much more traditional 
analyses, generally attempting to determine 


POTENTIAL FOR USE 


Discussions with leading health planners, ex- 
amination of the literature, and review of the 
findings of the OTA-sponsored empirical study 
mentioned above (358) yield the impression that 
capital budgeting procedures—specifically, life- 
cycle cost analysis—are one aspect of decision- 
making in planning which is receiving substan- 
tial emphasis. As noted previously, lifecycle 
cost analysis includes the notion of operating 
costs throughout the expected life of the invest- 
ment, as well as the capital depreciation sched- 
ule. Analysts argue that the total cost of a pur- 
chase includes the cost of operating it as well as 
buying it. Sophisticated financial and account- 
ing techniques are required to estimate deprecia- 
tion schedules, opportunity costs, and the costs 
of secondary effects such as might be caused by 
displacement of personnel or by increased/de- 
creased utilization in another part of the system. 


In effect, HRA and its Health Planning Cen- 
ters appear to be concentrating on the cost side 
of the ledger, as opposed to the effectiveness 
side. They appear to be telling the HSAs that so- 
cietal cost is much more than initial purchase 
price, and that as far as effectiveness is con- 
cerned, national, regional, or local standards 
and guidelines will have to suffice for now. This 
appears to be a reasonable approach, although 
even without good efficacy information, sec- 
ondary systems’ effects (e.g., lost revenue to the 
provider, changes in reimbursement rates, 
changes in physician practice patterns) could at 
least be described and their magnitude esti- 


whether a given service meets minimum utiliza- 
tion standards, standards which Kristein has 
characterized as resting more on equity than ef- 
ficiency (332). The national planning guidelines 
or similar standards are often used for these 
determinations. The assumptions are that meet- 
ing these standards satisfies basic medical 
“needs” and guarantees at least a minimal level 
of efficiency, and is therefore cost effective. Sel- 
dom, if ever, though, are costs weighed against 
expected health benefits. 


mated, as was done by one of the HSAs inter- 
viewed (see earlier discussion). 


Considering these secondary effects raises 
again the question of private versus public costs 
and benefits, and how these variables should be 
treated. The resolution of this issue has enor- 
mous implications regarding reimbursement 
rates, competitive systems development, and 
public responsibility vis-a-vis financial solvency 
of nonprofit as well as public health care institu- 
tions. As an example, if an HSA denies a com- 
munity hospital the right to operate a service on 
the grounds that certain utilization standards 
(e.g., national guidelines) are not met, thus 
causing the hospital to lose significant revenue, 
how should cost changes be treated? First of all, 
some utilization will be shifted to other facili- 
ties, which will now operate at a higher capacity 
and thus, presumably, more efficiently. But the 
losing facility will find itself with one less cost 
center with which to allocate overhead and will 


-have to absorb the now unallocated fixed costs 


elsewhere. What responsibility does the HSA 
(or society) have to the losing facility with re- 
spect to assuring that it recover its fixed costs? 
And how should the costs be treated in the anal- 
ysis? 


Furthermore, revenue-changing decisions in- 
evitably affect reimbursement rates. In Mary- 
land, for instance, the Rate Review Commission 
has been at odds with that State’s HSAs, be- 
cause the HSA-approved projects lend legiti- 
macy to requested rate increases (15). This class 
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of problems may always be present as long as 
facilities are reimbursed on a cost basis. Since 
one of the purposes of the HSAs’ activity is to 
ensure efficient utilization as measured by 
standards such as the national guidelines, the 
question becomes whether the HSAs are actual- 
ly capable of ensuring that such standards are 
met. The HSA’s control of utilization is indirect 
at best, and HSAs cannot easily fine-tune their 
approach. For that fine-tuning, Professional 
Standards Review Organizations (PSROs) mon- 
itor physician activities in the facilities, but even 
they only or primarily monitor certain Federal 
beneficiaries.* To help ensure that valid CEA/ 
CBAs are done and that their results are effec- 
tively used, therefore, it would seem desirable 
to coordinate the planning process (i.e., HSAs), 
the quality assurance/utilization review process 
(i.e., PSROs), and the reimbursement/rate re- 
view process. 


Earlier it was noted that the time available to 
an HSA staff for a CON analysis is very short, 
probably considerably less than 1 month. One 
recommended method to overcome this time 
constraint is to require the applicant to do the 
analysis (362). For a renovation replacement 
project, for instance, the planning agency might 
require the institution to provide an analysis of 
five alternatives: 1) reparation, 2) partial re- 
placement, 3) a decrease of capacity (or a small- 
er project), 4) an increase of capacity (or a larger 
project), or 5) no change. This technique does 
require tradeoffs to be considered, and it might 
allow the planning staff time to respond to an 
analysis in an appropriate manner and often to 
anticipate issues. 


CEA and Competition 


The private versus public cost controversy 
takes on great significance when the HSA and 
the HMO interact. In the example cited earlier, 
the HMO contended that since its costs would 
be lowered by providing the service itself rather 
than by contracting for it, the HMO would be at 
a better competitive advantage, thus forcing the 
fee-for-service sector, ultimately, to lower its 


5See ch. 6 of this volume for further discussion of PSRO activi- 
ties. 


prices or to face the possibility of losing sub- 
scribers. This argument—what is good for the 
firm is good for society—is the classic argument 
for the free market firm. The HSA countered 
with its own analysis showing that the addition- 
al service would result in overcapacity in the 
HSA area resulting in higher societal costs—a 
classic social welfare argument. 


Congress has apparently resolved this contro- 
versy with the passage of the 1979 Public Health 
Service Act Amendments. These amendments 
contain a provision forbidding HSAs to deny a 
CON to an HMO as long as the HMO can effec- 
tively demonstrate that the investment is benefi- 
cial to its own subscribers and that it is more ef- 
ficient for the HMO than other investments. 
That provision is coupled with the law’s require- 
ment that HSAs must consider “the strengthen- 
ing of competitive forces in the health services 
industry wherever competition and consumer 
choice can constructively serve . . . to advance 
the purposes of quality assurance, cost effective- 
ness, and access” (15). 


The new mandate contained in these amend- 
ments sets up potential forces that run counter 
to the conceptual basis for using CEA/CBA. 
The rationale underlying CEA/CBA method- 
ology is that the market is not operating prop- 
erly. The role of CEA/CBA in planning within 
the context of a competitive environment is un- 
clear. 


CEA and HSA Orientation 


Other problems with the use of CEA/CBA in 
health planning—apart from the conceptual dif- 
ficulties of using CEA/CBA within the construct 
of a partially competitive system—arise because 
it is unclear whether the HSA is (or should be) 
an agent of the community for which it plans, or 
an agent of the larger society. That distinction is 
important because the orientation which an 
analysis takes is very much dependent on the 
orientation of both the sponsor and the con- 
sumer of the analysis. 


OTA found that not only is there no stand- 
ard, generally accepted set of CEA/CBA meth- 
ods for all technologies, but also that many of 
the parameters chosen, as well as how they are 


valued, depend on the purpose of the analysis 
and for whom it is done. Consequently, the re- 
sults of even a technically superior CEA/CBA 
by an HSA could vary significantly depending 
on the importance which the agency attached to 
saving Federal tax dollars or to ensuring that its 
local community received all the health care 
which was needed, subject to prevailing guide- 
lines. ) 


One HSA, for example, may set its priorities 
in large part on the basis of total societal eco- 
nomic payoff rather than health “need” alone. 
Thus, an analysis by this HSA would include 
not only health benefits, but also economic 
benefits such as decreased health care costs and 
increased wages. This HSA may reject programs 
such as large hospital capital investments be- 
cause they seem to weigh too heavily on the cost 
side without sufficient demonstrable or proven 
benefits, opting instead for programs such as 
alcoholism programs that increase working 
hours. Another HSA may take the attitude that 
since its community health care system is not 
operating under a budget, it should consider 
“need” and efficiency in meeting that need 
alone. The board of that HSA may feel that in- 
creasing the productivity of the local working 
force is not relevant to its decision process, that 
is, including productivity variables will bias the 
selection process away from programs for its 
nonworking constituents. 


There is some conflicting evidence as to how 
the HSAs do view themselves, but generally it is 
believed that the more local orientation holds. 
HRA Administrator Foley stated that despite 
the law’s clear mandate to control costs, “from 
the perspective of HSAs and State agencies, it is 
not clear that their priority is on ‘reasonable 
costs’,” generally because of the third-party 
reimbursement system. Foley believes that “the 
major proponent of giving priority to the goal 
of reasonable cost is the Governors’ Offices and 
States’ legislatures” (211). 


Luft and Frisvold, on the basis of an intensive 
review of CON decisions of two CHP agencies, 
stated that, at least prior to 1974, “need” was 
the primary criterion on which decisions were 
based, and that an applicant could easily con- 
vince the agency that the proposed service was 
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“needed” even in the face of a surplus, as defined 
by established standards (375). Thus, Salkever 
and Bice were able to show that CON did not 
constrain overall investment in capital budget- 
ing, although it did alter the types of invest- 
ments made (530), the latter point possibly pro- 
viding a clue to the potential role of CEA/CBA. 


Most of the studies cited above analyzed the 
effects of the CON process immediately follow- 
ing its introduction. Recently published reports 
based on surveys completed by the American 
Health Planning Association, however, state 
that the CON process has yielded substantial 
savings (417). Although the surveys and thus 
their results have been questioned, this informa- 
tion indicates a continuing controversy over the 
effectiveness of CON. Altman, in attempting to 
describe incentives of planners, noted that the 
constituents of the local HSA were the local pro- 
viders and consumers, both of whom have eco- 
nomic incentives to increase health care re- 
sources in their own area, particularly, since 
such resources are ordinarily paid for on a much 
broader basis; the constituents of the State plan- 
ning agency, however, are the Governor and the 
State legislature, whose economic incentives are 
to constrain costs (15). Consequently, it is not 
unusual for State and local officials to disagree 
on health resource policy issues (100). Interest- 
ingly, however, roles are sometimes reversed, 
with local agencies being surprisingly cost con- 
scious—recommending against CON—and 
State agencies being surprisingly insensitive to 
cost—awarding CON over an HSA’s objections 
(15,189,499). 


Above it was noted that according to some 
studies, CON has resulted in a shifting of capital 
expenditures, but not in a decrease in total cap- 
ital expenditures. The purpose of CEA/CBA is 
not to constrain costs, but to assist in more effi- . 
cient allocation of resources, and if properly ap- 
plied, the CEA/CBA technique possibly could 
play a valuable role in the CON process. The 
first attempts to evaluate the effectiveness of 
CON initially focused on its superficial objec- 
tives (e.g., restraining increases in hospital 
beds). These studies indicated success. A second 
generation of more sophisticated analyses as- 
sumed that the real objective of CON was to al- 
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locate or to constrain capital investment and 
thus costs, and these analyses indicated failure 
(530). A third generation of analyses, which has 
not yet been performed to OTA’s knowledge, 
could attempt to determine whether the shift in 
resources caused by CON activity was cost ef- 
fective. That is, is society better off (more 
healthier, happier) now with, for example, so- 
phisticated diagnostic technology than it would 
be if more beds had been built? CEA/CBA in 
theory could shed light on this issue; the possi- 
bility of its being used in the future for this pur- 
pose, therefore, may merit consideration. 


A key question remaining is how should 
CEA/CBA be used in the planning process? To 
constrain costs or to obtain the best buy for the 
money? The evidence indicates that CEA/CBA 
is most effective when it is used within a con- 
strained budget, that is, when choices have to be 
made. As was previously noted, HSAs currently 
plan in an economic semivacuum. Although 
they are told to consider costs when they plan 
for need, HSAs plan for a health system which 
is under no budget constraint. In some cases, for 
instance, HSAs are urged to consider the “finan- 
cial feasibility” of a project; the stamp of 
approval (i.e., CON), however, is a ticket for 
financial feasibility, since once a CON is 
awarded, reimbursement (full cost recovery) is 
ordinarily guaranteed (100). This circular situa- 
tion had led to calls for either external regional 
budget controls or for the less stringent measure 
of requiring the agency to evaluate alternative 
projects to accomplish the stated objective 
(375). The former would require a major re- 
structuring of the country’s health care financ- 
ing system.® The latter, requiring alternative 
proposals, is therefore advocated as a more fea- 
sible solution, and indeed, is practiced by some 
HSAs interviewed for the OTA study (358). The 
latter mechanism by itself, however, is not suffi- 
cient as a cost-containment strategy since it 
allows the possibility of the acceptance of an 
unlimited number of objectives. In this case, a 
CEA/CBA would simply help the planner to en- 


*Altman believes this would further politicize the decisionmak- 
ing process (15); if such is the case, it would not be conducive to 
objective analysis. 


sure that each objective is accomplished in an ef- 
ficient manner, albeit helping to ensure that 
more benefit is obtained per dollar expended. 
By itself, however, use of CEA/CBA is not suf- 
ficient for containing overall health system 
costs. 


The resource allocation decisionmaking proc- 
ess of HSAs is basically a political process, 
rather than a technical one. The law envisions 
this implicitly by requiring that the controlling 
board be representative of local constituents. 
Nevertheless, there are some indications that, at 
least during the CON process—which is at pres- 
ent the primary means of affecting the system— 
the planners and their analysts are becoming 
more sophisticated at the capital budgeting 
process. There is little indication, however, that 
HSAs have attempted to estimate the health im- 
plications of their decision process; there failure 
to do this is understandable given the lack of 
knowledge among the medical and health com- 
munity about this. In general, decisions are 
based on either the national guidelines’? or on 
similar standards, both of which are themselves 
often based on “efficient” utilization rates and 
ordinarily set by consensus of experts without 
information on marginal costs and benefits. 


There does appear to be a serious effort in 
some instances to compare the cost of programs 
to the cost of alternative programs which ac- 
complish the same objective (e.g., to compare 
the cost of a surgicenter to the alternative cost of 
performing inpatient surgery). In these in- 
stances, benefits (or risks) are usually assumed 
to be equal across alternatives. If that assump- 
tion is true, then the study in which it is made 
can legitimately be called a CEA/CBA, on the 
net-cost end of the analytical spectrum. If not 
true, however, the study is simply a financial 
impact analysis. Most of the studies that have 
been done appear to be examples of financial 
impact analysis, because although equality of 
benefits across alternatives is often claimed, in 
fact, the analysts seldom have any hard data to 
prove it. 


"See app. B. 
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USE OF CEA/CBA IN HEALTH PLANNING: GENERAL FINDINGS 


The potential for increased use of CEA/CBA 
in health planning hinges on several elements: 


e the ability of HSA staffs to upgrade their 
technical skills and fundamental under- 
standing of CEA/CBA methods, and to in- 
crease the amount of time and financial re- 
sources available to conduct analyses; 

e the establishment of budgetary bounds 
within which investment decisions are com- 
peting with one another; 

e the extent to which market forces operate 
and are encouraged; 

e the ability to measure the health effects of 
technology and the availability of such 
data; 

e the availability of standard health care uti- 
lization data; and 

e the state-of-the-art of the CEA/CBA tech- 
niques themselves. 


Meanwhile, health planning agencies could 
perform analyses by explicitly listing or array- 
ing all elements which are included or will be af- 
fected by an investment. At the most basic level, 
this would be a descriptive process: When costs 
and effects can be quantified, that would be 


done; when they can be combined, that would 
also be done. If there is insufficient evidence but 
some probability of a cost or effect resulting 
from the decision, it would be described and 
commented on, but not necessarily quantified. 


This approach would require systematic anal- 
ysis but would not require as sophisticated ana- 
lytical skills. Since the health planning decision- 
making process is political in nature and ulti- 
mately rests on intangible factors anyway, fine- 
ly tuned studies with valid, aggregated, and 
quantified variables are not essential for the 
process to be assisted by CEA/CBA methodolo- 
gy. Moreover, as analysts become more familiar 
with formal CEA/CBA techniques, analyses 
could evolve toward increased sophistication. 


In this manner, the intent of the health plan- 
ning law might be met without requiring an im- 
mediate herculean effort to upgrade the tech- 
nical skills of planners and their analysts. This 
suggested approach could also assist in rational- 
izing the planning process and provide the foun- 
dation for the time when, and if, budget con- 
trols are ever imposed on the health care sys- 
tem. 
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8. 


Market Approval for Drugs 


and Medical Devices 


INTRODUCTION AND BACKGROUND 


The Federal Government has been authorized 
to regulate various aspects of drugs in the 
United States since President Theodore Roose- 
velt signed into law the Pure Food and Drug Act 
of 1906 (113). That law was in part intended to 
help prevent adulteration and misbranding of 
drug products. In 1912, Congress enacted the 
“Sherley amendment,” which prohibited com- 
panies from making false and fraudulent cura- 
tive or therapeutic claims on the labels of their 
products. In 1927, Congress established the 
Food and Drug Administration (FDA). 


The 1938 Food, Drug, and Cosmetic Act, 
passed by Congress in response to a tragic event 
in which over 100 people died from ingesting a 
lethal drug product,? increased the Federal Gov- 
ernment’s regulatory control over the marketing 
of drugs, devices, foods, and cosmetics. In addi- 
tion to establishing several labeling require- 
ments for drugs, the 1938 Act prohibited inter- 
state commercial shipment of new drugs until 
they had been adequately evaluated by the Fed- 
eral Government to show that they were safe 
under the conditions of use listed on their label. 
It also authorized FDA to remove from the mar- 
ket any drug it could prove to be unsafe. The 
Durham-Humphrey Amendment of 1951 de- 
fined criteria and categories based on levels of 
drug safety for restricting a drug to legend (i.e., 
prescription only) status. 


Again stimulated by a disaster, this one in- 
volving fetal abnormalities caused by the drug 
thalidomide,2, Congress enacted the Drug 


1Elixir of sulfanilamide made with the lethal solvent diethylene 
glycol. 

2Use of the sedative thalidomide by pregnant women in Europe 
and Japan resulted in several cases of phocomelia, a fetal abnor- 
mality characterized by flipper-like limbs. The drug was not ap- 
proved by FDA for marketing in the United States. 


Amendments of 1962 (Kefauver-Harris amend- 
ments). These amendments required drug man- 
ufacturers to provide “substantial evidence” 
that their products were efficacious as well as 
safe. The amendments also required drug man- 
ufacturers to report promptly to FDA informa- 
tion concerning the safety and efficacy of their 
marketed products and strengthened FDA's au- 
thority to remove unsafe or ineffective drugs 
from the market. In addition, the 1962 amend- 
ments authorized the notice of claimed inves- 
tigation for a new drug (IND) process—a legal 
mechanism used by FDA to regulate human in- 
vestigations of drugs. 


The Federal Government was given some au- 
thority to regulate medical devices in the 
original 1938 Food, Drug, and Cosmetic Act. 
Congress substantially increased this Federal 
regulatory authority, however, by passing the 
Medical Device Amendments of 1976 (116). The 
1976 amendments authorize FDA to require de- 
vice manufacturers to demonstrate acceptable, 
levels of safety and effectiveness for their prod- 
ucts. They establish three categories of prod- 
ucts, and each category has a different level of 
Federal control. Medical devices that are im- 
planted, used in supporting or sustaining human 
life, of substantial importance in preventing im- 
pairment of human health, or that pose a poten- 
tial unreasonable risk of illness or injury, for ex- 
ample, usually are subjected to premarketing 
evaluation of safety and efficacy. The 1976 
amendments permit FDA to ban or remove 
from interstate commerce unsafe or ineffective 
devices. They also allow the Agency to restrict 
the use of a device to persons with specific train- 
ing or experience or to those in specified facili- 
ties if the device’s safety and effectiveness can- 
not be assured in general use. 
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FDA’S PRESCRIPTION DRUG MARKET APPROVAL PROCESS 


Since 1962, FDA has used its statutory and 
regulatory authority to establish an extensive 
system for evaluating virtually every new pre- 
scription drug prior to its release into the gener- 
al medical marketplace. By law, FDA must base 
its evaluation of prescription drugs on two cri- 
teria: safety and efficacy.* The procedures used 
by FDA in this premarket approval process, de- 
scribed in detail elsewhere (119,144), are briefly 
summarized below. To initiate human investi- 
gations involving agents legally defined as “new 
drugs,” the drug “sponsor” (e.g., a manufac- 
turer or an independent investigator of a new 
drug) must obtain FDA's approval. To obtain 
such approval, the sponsor submits to FDA's 
Bureau of Drugs (BOD) an IND, describing the 
qualifications of the investigators and the design 
of the planned trials. In the IND review process, 
BOD also examines data regarding the phar- 
macology and toxicology of the applicant drug 
collected in animal studies and in human studies 
that were not subject to FDA approval (e.g., 
those conducted in foreign countries). If the 
sponsor's IND is approved, the sponsor may 
proceed with clinical testing (i.e., testing in 
human subjects) in the United States. After com- 
pleting clinical testing under IND, the sponsor 
files with BOD a new drug application (NDA) 
that describes in detail the results of IND clinical 
trials. The applicant drug has usually been 
tested in 500 to 3,000 human test subjects (306). 
By filing an NDA, the sponsor is requesting 
FDA's permission to market the new drug in in- 
terstate commerce. 


According to Dr. Marion J. Finkel, Associate 
Director for New Drug Evaluation, BOD, FDA, 
the review process for INDs and NDAs proceeds 
as follows (206): 


*Food, Drug, and Cosmetic Act S505, 21 U.S.C. S 355 (1976) 
part 314 (1979). The Act uses the term “effectiveness” instead of 
“efficacy.” 


Each IND and NDA application is reviewed 
by a team of FDA scientists: a physician, a phar- 
macologist, a chemist, a pharmacokineticist, 
usually a biometrician, and when applicable, a 
microbiologist. Important NDA’s are then pre- 
sented for consideration to advisory commit- 
tees, of which the Bureau of Drugs has 13, con- 
sisting of extramural [mostly nongovernmental] 
experts in, principally, the subspecialities of 
medicine, clinical pharmacology, and biomet- 
rics. The committees recommend to the FDA 
whether or not an NDA should be approved for 
marketing and, if so, under what labeling, and 
whether the sponsor should be requested to per- 
form additional studies in the postmarket- 
ing phase. When recommendations are made 
against approval, the committees provide advice 
on new studies to be done by the sponsors to ex- 
plore the drug's safety and effectiveness further. 
The use of advisory committees is the FDA’s pri- 
mary method for broadening the decisionmak- 
ing process. 


During the NDA review process, FDA re- 
viewers analyze the sponsor’s summaries of 
data—and when needed the actual raw data— 
generated from clinical investigations. A major 
task for BOD during this review process is to en- 
sure that clinical experimental data support the 
sponsor's claims for the drug’s safety. and ef- 
ficacy that appear in the drug’s labeling. The 
NDA review process usually entails delibera- 
tions between FDA and a drug’s sponsor regard- 
ing claims for safety and efficacy. If FDA con- 
cludes that an NDA is deficient, it usually will 
require the sponsor to generate new data, modi- 
fy its NDA with additional information, and re- 
submit the application to the Agency. Once a 
sponsor obtains an NDA approval from FDA, it 
is authorized to market the drug in interstate 
commerce for only the specific indications (uses) 
that have been authorized by FDA. 
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FDA’S MEDICAL DEVICE EVALUATION PROCESS 


The Medical Device Amendments of 1976 
(Public Law 94-295) require FDA's Bureau of 
Medical Devices (BMD) to classify each medical 
device—on the basis of the level of regulation 
necessary to ensure safety and efficacy—into 
one of three regulatory classes. Products placed 
in the class I category—those requiring 'the least 
controls to ensure their safety and efficacy—are 
subject only to general controls, including pre- 
market notification, registration of the man- 
ufacturer, prohibition of product misbranding 
or adulteration, adherence to FDA-promulgated 
good manufacturing practices, and compliance 
with various recordkeeping requirements. Class 
II products—those for which general controls 
are deemed inadequate to ensure their safety 
and efficacy—must comply with performance 
standards either established or recognized by 
BMD when existing information permits devel- 
opment of such standards. General controls also 
apply to class II devices unless superseded by a 
standard. Class III devices—those for which 
neither general controls nor performance stand- 
ards alone are sufficient to ensure their safety 
and efficacy—are subject to premarketing ap- 
proval. (BMD can also require premarketing ap- 
proval of devices for which insufficient informa- 
tion is available for the development of ap- 
plicable performance standards.) Class III de- 
vices are subject to general controls. 


SAFETY AND EFFICACY CRITERIA 


The Food, Drug, and Cosmetic Act mandates 
FDA to require a drug sponsor to: 1) collect, by 
all methods reasonably applicable, data that 
demonstrate a drug’s safety, and 2) generate 
“substantial evidence” from controlled trials to 
show that the drug is efficacious for use under 
the conditions set forth in the proposed labeling 
(144). Although the Act provides no definitions 
and little guidance for the meanings of safe and 


The process BMD uses to evaluate the safety 
and efficacy of new class III medical devices is 
similar to that used by BOD in its NDA reviews. 
BMD, however, is required by law to use its ad- 
visory panels during a product's review, 
whereas BOD’s use of advisory panels is discre- 
tionary. BMD is also authorized to use another 
premarketing approval process, the product de- 
velopment protocol (PDP). PDP was designed 
to encourage the development of, and to stream- 
line the approval process for, innovative medi- 
cal devices. During a PDP process, investigation 
of a device and the development of information 
necessary for its premarket approval are merged 
into one regulatory mechanism. BMD works 
closely with the manufacturer while informa- 
tion to support the device's safety and effec- 
tiveness is being developed. The 1976 Medical 
Device Amendments also include an investiga- 
tional device exemption (IDE) provision. IDE 
allows FDA (technically, the Secretary of 
Health and Human Services) to exempt a new 
device from other provisions of the amendments 
to permit controlled testing of new devices prior 
to commercial marketing. The IDE process is 
similar to the IND process that BOD uses.‘ 


445 FR 3732. 


effective,> it does describe “substantial evi- 
dence” of effectiveness as follows (144): 


SOTA differentiates “efficacy” and “effectiveness” as follows 
(465): 
Efficacy: The probability of benefit to individuals in defined popu- 
lations from a medical technology applied for a given medical 
problem under ideal conditions of use. Effectiveness: Same as ef- 
ficacy except that it refers to“. . . average conditions of use.” 

Although the FDA statutes use the terms “effective” and “effec- 
tiveness,” the definitions implicit in the statutes appear to mean 
“efficacy.” Thus, “efficacy” is used in the present report. 
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The term “substantial evidence” means evi- 
dence consisting of adequate and well-controlled 
investigations, including clinical investigations, 
by experts qualified by scientific training and ex- 
perience to evaluate the effectiveness of the drug 
involved, on the basis of which it could fairly 
and responsibly be concluded by such experts 
that the drug will have the effect it purports or is 
represented to have under the conditions of use 
prescribed, recommended, or suggested in the 
labeling or proposed labeling thereof. 


Because no drug is absolutely safe or always 
effective under all conditions of use, FDA 
weighs a drug’s benefits in comparison to its 
risks. In FDA's evaluation process, an applicant 
drug’s safety and efficacy are compared to the 
safety and efficacy of approved products or 
medical procedures used to achieve similar ther- 
apeutic objectives. Comparisons are also made 
with the effects of inert substances (placebos). 
According to Dr. Marion J. Finkel of FDA (204): 


A drug cannot be considered safe if it is less ef- 
fective and has more side effects than other 
drugs labeled for the same indication. It might 
be safe for certain subpopulations or certain re- 
stricted indications, however. 


According to Dr. Robert Temple of FDA (594): 


We're not supposed to refuse approval be- 
cause of lesser effectiveness, but we can on 
grounds of lesser safety which reduces the bene- 
fit/risk ratio. Relative effectiveness can be noted 
in labeling. 


FDA also uses such comparisons when selecting 
NDAs for its priority review or “fast track” re- 
view process. 


FDA has published 25 clinical guidelines that 
define the types of studies it deems appropriate 
to use to establish safety and efficacy of a drug. 
The performance criteria used to evaluate an in- 
dividual drug’s safety and efficacy, however, 
can vary among different pharmacological 
classes. This variance results from the imprecise 
nature of pharmacologic intervention in disease. 
The benefits and risks of a drug can vary from 
disease to disease, from population to popula- 
tion, and from clinical situation to clinical situa- 
tion. Drug-induced hair loss, for example, is 
often viewed as an acceptable risk of taking a 
drug that is an effective treatment for cancer, 
especially when no other effective therapy ex- 
ists. The same side effect, however, would prob- 
ably not be acceptable for a drug that reduces 
the severity of a minor, self-limiting condition. 


The complete risk-benefit analysis of a drug, 
therefore, is not based solely on statistical 
evaluation of data on safety and efficacy gener- 
ated from clinical trials but also on “the context 
of the disease for which [the drug] is intended, 
the availability of other therapeutic modalities, 
including other forms of pharmacologic thera- 
py, and public health implications of its avail- 
ability” (144). 


The task of evaluating the safety and efficacy 
of drugs, as well as FDA's ability to perform this 
task, has been subjected to extensive analysis 
and debate (119,144,242,613). 


CURRENT ECONOMIC CONSIDERATIONS IN FDA’S DRUG AND 
MEDICAL DEVICE APPROVAL PROCESSES 


FDA does not directly use economic criteria 
in its approval processes for drugs and medical 
devices. The Agency’s primary statutory au- 
thority, the Food, Drug, and Cosmetic Act, nei- 
ther authorizes nor prohibits the use of econom- 
ic criteria in FDA's evaluation of applicant 
drugs and devices. The legality of using cost ef- 
fectiveness to help evaluate new drugs and de- 
vices has not been tested. 


Although FDA does not formally assess the 
potential economic impact a drug or medical de- 
vice might have on the allocation of health care 
resources, some FDA actions may be based indi- 
rectly on—or taken in spite of—economic con- 
siderations. Further, some FDA actions certain- 
ly have economic impacts. Several examples are 
cited below. 


Classifying the Potential Therapeutic Impor- 
tance of New Drugs: FDA sets priorities for its 
review of new drugs according to the potential 
therapeutic usefulness of each applicant drug. 
This priority-setting process supposedly begins 
within 6 months after a sponsor submits an IND 
to FDA (206). On the basis of preliminary in- 
formation about a drug’s pharmacological ef- 
fects and on data from clinical trials, a drug is 
assigned a number and a letter derived from the 
following classification schemes (221): 


Numerical Classification 


1= New molecular entity not previously 
marketed in the United States. 

2= New salt, ester, or derivative of an active 
moiety marketed in the United States. 

3= New formulation of a compound mar- 
keted in the United States. 

4= New combination of two or more com- 
ponents not previously marketed togeth- 
er in the United States. 

5= Product duplicates a drug marketed by 
another firm in the United States. 

6= Product already marketed by some firm 
in the United States; approval being 
sought for new indication of use. 


Letter Classification 


A= major therapeutic advance over other 
currently available drugs, etc. 

B= modest (or moderate) therapeutic ad- 
vance over other currently available 
drugs, etc. 

C= little or no appreciable therapeutic ad- 
vance over other currently available 
drugs, etc. 


The purpose of using a drug classification 
scheme of this type is to expedite the review of 
data for drugs that represent important new 
therapeutic entities. FDA seeks to review NDAs 
submitted for drugs it assigned a 1A or 1B rating 
as expeditiously as possible; such reviews re- 
ceive high priority by BOD (206). 


Even though the criteria used to classify a new 
drug are based on scientific data relating to a 
drug’s clinical safety and efficacy, there are in- 
direct economic consequences of FDA’s selec- 
tion process. The expeditious review and result- 
ant early marketing of a new drug that repre- 


66-220 0 - 80 - 7 
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sents a major therapeutic breakthrough in the 
treatment of a heretofore uncontrollable disease 
could help reduce the use of existing treatment 
measures, such as hospitalization and surgery. 
The recently introduced drug cimetidine, for ex- 
ample, appears to provide safe, effective, and 
relatively inexpensive (compared to hospitaliza- 
tion or surgery) treatment for duodenal ulcers; 
using cimetidine, some ulcer patients may avoid 
hospitalization (203,632). FDA rated cimetidine 
as a 1A drug. If indeed the use of cimetidine 
reduces ulcer patients’ hospitalizations and 
surgery, then an expeditious FDA review and 
approval of the drug could help reduce medical 
expenditures for the treatment of duodenal 
ulcers. A complete analysis would include cal- 
culations of the potential economic impacts of 
delaying the review of one or more other INDs 
or NDAs, in order to expedite cimetidine’s ap- 
plication review. 


According to FDA’s Dr. Marion J. Finkel 
(205): 


NDAs for 1A and 1B drugs are full NDAs, 
containing all of the safety and efficacy data re- 
quired for any NDA. Expeditious review of 
these merely means that the NDAs do not wait 
their turn in the pipeline but are reviewed before 
NDAs with lesser classifications . . . 

Rarely, FDA will accept an NDA for a 1A 
drug without as much long term human (or ani- 
mal) safety data required for NDA approval. 


In at least some cases, however, the expedi- 
tious review of an important new drug might re- 
sult in an incomplete assessment of the drug’s 
safety, and an unexpectedly significant level of 
adverse reactions to the drug might occur. To 
help prevent such an occurrence, FDA can—and 
sometimes does—ask a sponsor to conduct post- 
marketing surveillance as a condition of ap- 
proval for a new drug (131). 


Use of Abbreviated New Drug Applications 
(ANDA) and “Paper” NDAs: Two examples of 
the indirect use of economic considerations in 
FDA’s drug approval processes are the ANDA 
and the so-called “paper” NDA. The ANDA 
process enables a drug manufacturer to obtain 
from FDA marketing approval for a “generic” 
drug product that is purported to be an identical 
version of an already approved product, usually 


90 © The Implications of Cost-Effectiveness Analysis of Medical Technology 





after the originator’s patent has expired. Be- 
cause of FDA's earlier interpretation of the 
Food, Drug, and Cosmetic Act, ANDAs are cur- 
rently only used for products originally ap- 
proved by FDA between 1938 and 1962. FDA is 
preparing a policy, however, that will permit 
the use of ANDAs for products approved after 
1962. A manufacturer can obtain an ANDA es- 
sentially by demonstrating that it complies with 
FDA's current good manufacturing practices 
and labeling requirements and that it can make 
a product that is at least chemically equivalent 
and supposedly bioequivalent to the origina- 
tor’s. Although evidence of chemical equiva- 
lence is always required, the FDA Commission- 
er can forgo requiring evidence of bioavailabil- 
ity in the ANDA process for a given drug. 


“Paper” NDAs are designed to permit a drug 
manufacturer to meet the requirements for sub- 
mitting evidence of the safety and efficacy of its 
post-1962 product by citing existing data from 
published clinical trials involving a chemically 
equivalent, previously approved product. The 
legality of “paper” NDAs was challenged in 
court by the Pharmaceutical Manufacturers As- 
sociation (PMA) and a few major drug manu- 
facturers (473). The court dismissed the suit on 
the grounds that the plantiffs had not exhausted 
their administrative remedies within FDA, so 
PMA has filed a petition that: 1) questions 
FDA's authority to implement a “paper” NDA 
policy, and 2) requests that if such a policy is 
implemented by FDA, it be done through the of- 
ficial notice and comment rulemaking pro- 
cedure. 


The primary purpose of both the ANDA and 
“paper” NDA modifications in the new drug ap- 
proval process is to facilitate the marketing ap- 
proval of drug products identical to those that 
FDA has previously found to be safe and effica- 
cious on the basis of data supplied in a product's 
original NDA. These mechanisms are designed 
to prevent unnecessary work burdens on FDA's 
resources. The use of ANDAs and “paper” 
NDAs is supposed to increase the availability of 
FDA staff to review NDAs submitted for inno- 
vative drugs, e.g., those categorized by FDA as 
1A. The use of these mechanisms also eliminates 
the need for sponsors to conduct duplicative 


clinical trials. Thus, manufacturers’ expenses 
associated with entering some existing drug 
product markets can be reduced. 


Removal of Diethylstilbestrol (DES) From 
Animal Feed: The estrogenic compound DES 
was used by poultry and other livestock raisers 
for many years to increase the muscle and fat 
content of their animals. FDA first approved 
such use of DES in 1954.° After that, DES was 
used in animal feed to reduce the total feed 
intake necessary to achieve maximum animal 
weight gain. Furthermore, some studies showed 
that the use of DES in animal feed helped reduce 
the cost of raising poultry and other livestock. 
Theoretically, therefore, the use of DES may 
have helped contain poultry and meat prices for 
consumers. 


Ingestion of DES by humans, however, was 
eventually correlated with an above-normal risk 
of developing certain types of cancer, such as 
adenocarcinoma of the vagina or the cervix.’ 
Because of DES’s cancer-producing potential, 
FDA attempted several times to remove this 
substance from animal feed. 


The authority for its efforts was the Delaney 
clause,* a proviso added to the 1958 food addi- 
tive amendment to the Food, Drug, and Cos- 
metic Act: 


. . no additive shall be deemed to be safe if 
found to induce cancer when ingested by man or 
animal... 


This clause was reiterated in the 1968 animal 
drug amendments to the same Act.? Under the 
animal drug amendments, approvals for the use 
of new animal drugs, which are used in the live- 
stock industry for the treatment and prevention 
of disease and as growth promoters, are to be 
granted only after a two-part evaluation by 
FDA. First, FDA must determine that the drug is 
safe and effective for use in animals. Second, it 
must assess the safety of potential residues that 
might occur in food derived from treated ani- 
mals.?° 
°44 FR 54853. 


744 FR 54852. 

*Sec. 409(c)(3)(A), Food, Drug, and Cosmetic Act, 21 U.S.C. 
348(c)(3)(A). 

*Sec. 512, 21 U.S.C. 360b. 

10Sec. 512, 23 U.S.C. 360b(d)(1). 
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After evaluating and finding DES to be a car- 
cinogenic substance, in 1972, FDA attempted to 
ban its use in animal feed. That effort termi- 
nated in a litany of legal proceedings with the 
livestock industry and animal feed manufactur- 
ers.!1 On January 12, 1976, therefore, FDA once 
again initiated a legal process to remove DES 
from animal feed. This time, the Agency also 
issued an inflation impact statement regarding 
its proposed ban of the use of DES.!? The state- 
ment concluded: 


1. There are no satisfactory alternatives to 
the Agency's proposed action which are 
consistent with the legal constraints im- 
posed by the Federal Food, Drug, and 
Cosmetic Act as amended. 

2. Operating expenses to feedlot producers 
of cattle for feed and other items are esti- 
mated to decrease by $156 million during 
the first year following a DES ban. These 
increased costs are expected to fall sub- 
stantially over the longer term as substi- 
tutes to DES become available in greater 
quantities. 

3. Retail prices of beef are estimated to rise 
by about 2 cents per pound, meaning the 
per capita cost of beef to consumers at 
current levels of consumption would in- 
crease from $2 to $3 annually. The aggre- 
gate consumer impact is estimated at $503 
million. 

4. A ban on DES would not cause major in- 
flation impacts, as defined by the HEW/ 
OMB criteria, in the areas of competition, 
productivity, supplies of materials, or use 
of energy. 

5. The benefits from implementing the pro- 
posed action will be the elimination of any 
risk of any cancer associated with the con- 
sumption, via the edible tissues of food- 
producing animals, of residues attributa- 


ble to DES. 


According to former FDA Chief Counsel, 
Richard Cooper, FDA’s mandate under the Del- 
aney clause and the 1968 animal drug amend- 
ments prevented the Agency from considering 
any economic benefits of DES in its decision to 
ban the drug from use in animal feed (127a). 


1144 FR 54854. 
2EDA docket No. 76N-002, Inflation Impact Statement DES, 
Office of Planning and Evaluation, January 1976. 


On June 29, 1979, then FDA Commissioner 
Donald Kennedy ordered the withdrawal of ap- 
proval of the new animal drug application for 
DES.'? In taking this action to ban the use of 
DES, Commissioner Kennedy stated: 


FDA is not authorized, under the Food, Drug 
and Cosmetic Act, in considering the question of 
whether a new animal drug has been shown to 
be safe for use, to weigh the “socio-economic” 
benefits that that drug provides against a health 
risk to the ultimate human consumers of treated 
animals. Even were I to attempt to weigh the 
benefits of DES against its risks, this record 
would not provide sufficient information to 
compute the risk associated with DES or to de- 
termine whether, and to what extent, use of DES 
provides any health benefit or even any econom- 
ic benefit to society. 


This case study illustrates one situation in 
which FDA interpreted the Delaney clause of the 
Food, Drug, and Cosmetic Act (its primary stat- 
utory authority) to mean that in its decisions 
concerning the removal of carcinogenic sub- 
stances from the market, it cannot consider the 
results of economic analyses. 


Evaluation of an X-Ray Equipment Perform- 
ance Standard Established by the Bureau of Ra- 
diological Health (BRH): This case illustrates 
FDA's use of cost-benefit analysis to evaluate 
one of its regulations. 


In 1972, FDA’s BRH promulgated regulations 
containing a performance standard for medical 
diagnostic X-ray systems and their major com- 
ponents.’ These regulations, among other 
things, required X-ray equipment manufactur- 
ers to certify that all specified components man- 
ufactured after August 1, 1979, comply with the 
new standard. Another provision stated that 
after August 1, 1974, no uncertified components 
could be assembled or reassembled into an X- 
ray system.'* The purpose of these regulations 
was to help ensure the radiation safety of diag- 
nostic X-ray systems.'° 


In 1974, BRH estimated the costs and benefits 
associated with these regulations. The Bureau 


1344 FR 54900. 

1421 CFR 1020.30, 1020.31, 1020.32. 
1521 CFR 100.16. 

1644 FR 22755. 
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estimated, for example, the expected numbers 
and costs of X-ray systems which would be mar- 
keted under the new performance standards. 
Upon reviewing 3 years of experience with the 
regulations for diagnostic X-ray systems, BRH 
realized that their earlier estimates of certain 
costs and benefits were incorrect. In 1978, using 
data not previously available, BRH conducted a 
second analysis of costs and benefits derived 
from the 1972 regulations. On the basis of this 
second analysis, BRH concluded:?” 


1. The provisions of section 1000.16 that are 
effective after August 1, 1979, will affect 
only a small fraction of the uncertified X- 
ray systems currently in use. 

2. The total impact of this regulation, in 
terms of increased cost for the X-ray 
equipment or interruption of health care 
delivery, may be significant. 

3. For those uncertified systems that would 
be affected, by virtue of their sale and re- 
location, section 1000.16 is not likely to 
be a cost-effective approach to improve 
the radiation safety performance of X-ray 
systems. 


Based on these conclusions, BRH, through the 
rulemaking process, modified its earlier regula- 
tions by, among other things, permitting the in- 
stallation of uncertified components into exist- 
ing systems whose components are all uncerti- 
fied and permitting the continued reassembly of 
uncertified equipment. It also clarified certain 
aspects of its performance standard for X-ray 
systems. '® 


17Ibid. 
1844 FR 49667. 


Incorporating Costs-Savings Studies Into 
Vaccine Approval Decisions: In its recently 
published report on viral and rickettsial vac- 
cines, FDA’s Panel of Review of Viral and Rick- 
ettsial Vaccines identified economics as a major 
consideration in the evaluation of vaccines.?9 
The panel noted that because adequate data are 
often lacking, economic considerations are 
often ignored in such evaluations. 


The panel stated: 


These data would contrast the cost of a vac- 
cine and its administration plus the costs (medi- 
cal care, rehabilitation, impairment of ability to 
earn income) of vaccine-related disease with 
costs of a similar nature that would have ac- 
crued from cases of the natural disease. 


The panel further stated: 


For an “acceptably safe” and effective vaccine 
against a serious disease, . . . the ratio should 
be highly favorable. However, if the preventable 
disease occurs chiefly in young children and is 
infrequently associated with permanent seque- 
lae, a different answer might result. The ques- 
tion then might become “how much cost can be 
justified to prevent one crippling or lethal case of 
disease?” This clearly requires societal rather 
than scientific judgment. 


This panel has strictly an advisory capacity 
within FDA, and its reports do not reflect offi- 
cial FDA policy. The panel's discussion of eco- 
nomics in the vaccine evaluation process, how- 
ever, does illustrate some concern among FDA’‘s 
advisors for evaluating the potential economic 
impacts of vaccines. 


1945 FR 25665. 


POTENTIAL APPLICATION OF CEA/CBA 


In order to facilitate the discussion presented 
in the next section of this chapter on the implica- 
tions of including cost-effectiveness criteria in 
the market approval process, OTA has devel- 
oped the hypothetical model of a cost-effective- 
ness analysis (CEA) outlined below. It is very 
important to keep in mind that the use of the 
model for illustrative purposes does not negate 
or diminish any of the weaknesses or possibil- 
ities for misuse involved in formal cost-effec- 


tiveness analyses/cost-benefit analyses (CEA/ 
CBAs) that seek a single, quantitative bottom- 
line, such as a cost-effectiveness ratio. Such a 
ratio is used in the model because 1) it points out 
some of the data and other problems related to 
use of bottom-lines, and 2) it is possible, given 
the quantitative nature of FDA's regulatory 
process, that use of a CEA ratio might be seri- 
ously considered by the agency if CEA were 
added to its mandate. 


OTA’s model is simplistic by intent and does 
not address the numerous assumptions and vari- 
ations that are possible. Several caveats and 
assumptions should accompany an analysis 
such as the one presented. Potential method- 
ological problems associated with the use of 
CEA in general are explained in detail in a 
background paper of this assessment.?° Addi- 
tional problems with its use in market approval 
processes are explained below. One problem, 
for example, is that data for some important 
variables would be difficult, if not impossible, 
to obtain. 


In assessing a drug’s cost effectiveness, FDA 
might first assess the agent's efficacy and safety 
and quantify its effects into measurable units of 
“net health effect.” Let us assume, for example, 
that an NDA is submitted for applicant drug 
“D,” which is used to treat high blood pressure 
(hypertension). Let us also assume that in pre- 
marketing clinical trials, drug “D” consistently 
lowered by 10 percent blood pressure in 50 per- 
cent of tested hypertensive patients, and that 
such a drop in pressure could be correlated with 
a 5-percent reduction in morbidity and mortali- 
ty (e.g., heart attacks, strokes, and kidney dis- 
ease). Suppose drug “D” also produced undesir- 
able side effects that in premarketing clinical 
trials accounted for a 2-percent increase in days 
of disability in the tested population. Let us also 
assume that the tested population accurately 
represents the general hypertensive population. 


To calculate the “net health effect” of drug 
“D,” all such positive effects (e.g., 5-percent 
reduction in mortality) and negative effects 
(e.g., 2-percent increase in disability caused by 
side effects) would have to be converted into a 
uniform and measurable unit of health.” The 
“net health effect’ of drug “D” could then be 
calculated for each indication listed on the pro- 
posed drug labeling (e.g., treatment of moderate 
hypertension in ambulant patients). Specific 
conditions of use (e.g., in conjunction with 
other drugs) and peculiar effects in special pop- 


2°Background Paper #1: Methodological Issues and Literature 
Review, prepared by OTA. 

21As discussed in Background Paper #1: Methodological Issues 
and Literature Review, some analysts have used the unit termed 
quality-adjusted life year for this purpose (652). 
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ulations (e.g., the elderly) could be accounted 
for in the calculation of net health effects. 


Once the drug’s effects were converted into 
measurable units of “net health effects,” the ‘net 
cost” of achieving a desirable level of health ef- 
fect (e.g., a 5-percent reduction in mortality and 
morbidity) would have to be calculated. A “net 
cost” could include such items as the cost of pur- 
chasing the drug, the cost of treating drug-in- 
duced side effects, and perhaps the cost of treat- 
ing other illnesses in the persons whose lives are 
saved by the use of drug “D.” Subtracted from 
such costs could be savings resulting from any 
reduced costs of hypertension treatment (e.g., 
lower use of previously approved drugs or de- 
cline in hospitalizations and in physician office 
visits) resulting from the use of applicant drug 


lw mr 
. 


At the conclusion of this phase of the anal- 
ysis, one could construct a ratio of net cost in 
dollars to one unit of “net health effect” 
achieved through the use of drug “D” in the 
treatment of hypertension, e.g.: 


Net cost 


Cost-effectiveness ratio = i 
Unit of net health effect 


In the next phase of the analysis, the net cost 
(in dollars) of achieving a desired net health ef- 
fect through the use of drug “D” would be com- 
pared with the net cost of achieving the same net 
health effect by using an existing approved 
treatment modality (e.g., another drug, sur- 
gery, or biofeedback) to lower blood pressure. 
Such a comparison of costs would require that 
cost-effectiveness ratios, i.e., net cost (in dol- 
lars)/unit of net health effect, be derived for 
each hypertension treatment modality. The fol- 
lowing hypothetical ratios for example, might 


be derived: 


Hypertension treatment Net cost/uniform unit 


modality of net health effect 
Applicant drug “D”.:.. $400 
Approved drug “A” ... $250 
Surgical procedure “X” . $3,000 
Biofeedback’ ?.../ 2507. $100 


Let us assume that no other treatment of hyper- 
tension was available at the time of the analysis. 


The final phase of such an analysis would be 
to establish criteria for judging the cost-effec- 
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tiveness ratios for each treatment modality and 
for using such ratios to help determine if appli- 
cant drug “D” should be approved. In the exam- 
ple above, if a criterion for market approval 
were that the applicant drug had to produce one 
unit of “net health effect” at a cost lower than 
the cost of using any other approved antihyper- 
tensive drug, then drug ‘“D” might not be ap- 
proved. If such a criterion included comparison 
with other approved forms of hypertension 
treatment, then drug “D” would presumably be 
approved if compared with surgery, although it 
might not be approved if compared with bio- 
feedback. When one considers in this hypotheti- 
cal evaluation process the need for a variety of 
available treatment modalities to meet individ- 
ualized patient situations, then the relatively 
small differences in cost effectiveness between 
applicant drug D, biofeedback, and approved 
drug A become much less important. 


Cost effectiveness might be used in formal re- 
view processes of drugs that have been mar- 
keted for a number of years, instead of those for 
new drugs. FDA conducts at least three such re- 


view processes, and to date, the Agency has not 
used cost effectiveness in any of them. First, in 
its drug efficacy study implementation project, 
FDA is conducting a one-time review of the ef- 
ficacy of drug products approved between 1938 
and 1962 (131). The Agency categorizes prod- 
ucts according to their documented clinical ef- 
ficacy, has removed from the market some 
products for which efficacy documentation is 
lacking (e.g., selected fixed-dosage antibiotic 
combination products), and is attempting to re- 
move other such products. Second, in its Over- 
the-Counter (OTC) Drug Review Program, 
FDA is conducting a one-time review of the safe- 
ty and efficacy of several hundred ingredients 
used in OTC or nonprescription drug products. 
Third, FDA (technically, the Secretary of Health 
and Human Services) has authority to remove 
from the market a drug that represents an “im- 
minent hazard” to the public’s health. When this 
authority was exercised to remove the antidia- 
betic drug phenformin from the market, FDA 
extensively reviewed the drug’s safety and to 
some extent its efficacy. 


IMPLICATIONS OF CEA/CBA IN MARKET APPROVAL: 


GENERAL FINDINGS 


As described in chapter 2, CEA is an analyti- 
cal device that can be used to help decision- 
makers allocate resources, usually in the public 
sector. The primary purpose of FDA’s drug and 
medical device market approval processes is to 
protect the public from unsafe and ineffective 
products. FDA’s marketing approval processes 
do, however, indirectly influence the allocation 
of public resources in at least two ways. First, 
the types of regulations and procedures estab- 
lished by FDA affect the allocation of its operat- 
ing budget. Second, many—if not most—of the 
drugs and medical devices approved by FDA 
eventually are used in health care services that 
are paid for through publicly financed pro- 
grams, such as medicare and medicaid. 


The use of cost effectiveness as a criterion in 
FDA's market approval processes for drugs and 
medical devices would require a substantial shift 


in the Federal Government's approach to regu- 
lating the medical care marketplace. At present, 
no Government effort is designed explicitly to 
reduce medical expenditures by directly slowing 
down or stopping the market introduction of 
medical goods and services. The Federal Gov- 
ernment does not extensively evaluate the cost 
effectiveness of drugs or medical devices in any 
of its efforts to contain health care costs. Con- 
ceivably, such evaluation could take place in the 
National Center for Health Care Technology, 
which advises the Health Care Financing Ad- 
ministration (HCFA) on reimbursement policies 
for selected medical technologies. Results of 
CEAs involving drugs and medical devices 
could be incorporated into HCFA’s policies for 
reimbursing the use of selected drugs and medi- 
cal technologies under medicare, medicaid, and 
any other federally operated health insurance 
program. 


EEE —_ 


Potential Positive Effects 


There are at least two hypothetical positive 
effects of incorporating CEA into FDA’s market 
approval process for drugs and devices. 


First, manufacturers could formally incorpo- 
rate cost-effectiveness criteria, based on societal 
values, into their research, development, and 
marketing strategies. Manufacturers most likely 
do use some form of CEA/CBA to allocate their 
R&D expenditures. The primary criteria used in 
such allocations, however, may emphasize such 
items as total sales, market portions, and re- 
turn-on-investment. If FDA used criteria such as 
reduced treatment costs, improved levels of 
health, and improved efficiency in disease pre- 
vention or treatment, then more manufacturers 
might develop new products and seek new mar- 
kets where existing treatment or prevention 
measures are ineffective or inefficient. Some in- 
dustry representatives claim that the leading 
drug research firms currently include such pub- 
lic health criteria in their research priorities. The 
existence of orphan drugs (i.e., existing agents 
for which there is a small, demonstrated clinical 
need but no manufacturer), however, indicates 
that at least for certain products, profitability 
takes precedence over societal need. Conceiv- 
ably, such use of CEA could help the Govern- 
ment encourage participation by drug and medi- 
cal device manufacturers in public efforts to de- 
velop more cost-effective medical technologies. 


Second, if FDA were able to accurately assess 
the cost effectiveness of an applicant medical 
device or drug, the Agency might help reduce 
expenditures for inefficient products by keeping 
them off the market entirely. By evaluating the 
cost effectiveness of medical devices and drugs 
in the market approval process, the Govern- 
ment would be assessing a product very early in 
its diffusion process. Perhaps this early evalua- 
tion process would lead to better direction—and 
perhaps an expansion—of experimentation with 
new drugs and devices. At present, substantial 
non-FDA-approved clinical experimentation 
with newly approved drugs takes place in un- 
controlled situations. Such early evaluation, 
however, would have no effect on the inefficient 
use of drugs and medical devices found to be 
cost effective in clinical trials. 
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Potential Problems 


Among the consumer advocates, FDA em- 
ployees, and representatives of the pharmaceu- 
tical industry surveyed in this assessment, there 
appeared to be widespread agreement that cost 
effectiveness is not an appropriate criterion to 
use in the drug and medical device approval 
process. Using information and ideas obtained 
from several individuals, OTA developed the 
following analysis of potential problems. 


First, the market approval process may be too 
early in the life of a drug or medical device to 
evaluate its cost effectiveness; information re- 
garding a product's safety and efficacy are usu- 
ally available for only one or two indications of 
use at the time a sponsor submits an initial NDA 
for a new drug or an application for approval of ~ 
a new device. Data regarding a drug’s safety and 
efficacy in medical conditions not listed in the 
product's official labeling are not often gener- 
ated until a product has been marketed for at 
least a few years. If a new drug or medical de- 
vice were not approved for marketing because 
its cost effectiveness did not compare favorably 
with already approved products, then the new 
product might never be fully evaluated, particu- 
larly in the treatment of medical problems other 
than the one(s) studied initially. The total bene- 
fits, risks, and cost effectiveness of drugs that 
are used in the treatment of more than one medi- 
cal problem often may not be known for several 
years after the product's initial development. 
Examples of such products include propranolol 
used in the treatment of angina, migraine head- 
aches, and hypertension; selected antibiotics 
used in the treatment of acne; amantadine used 
in the treatment of parkinsonism; and phenyto- 
in used in the treatment of certain cardiac ar- 
rhythmias. In addition, rare or delayed-onset 
adverse drug reactions, for example, drug-in- 
duced cancers, would not likely be included in 
premarketing CEA. 


James B. Russo, SmithKline Corp., offers an 
industrial concern about the use of CBA in the 
market approval process for drugs (529): 


Our primary concern is over the fact that cost- 
benefit analysis in the drug area is a new disci- 
pline, and one which simply must not be widely 
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applied, particularly in the area of new drug ap- 
proval, until we have real confidence in its pre- 
dictive reliability . . . I don’t really know how 
to state that point as strongly as I'd like to. 
Think back to probenecid. The drug was devel- 
oped to slow the excretion of penicillin from the 
kidney, because in those days penicillin produc- 
tion was inadequate. By the time probenecid 
was shown to be safe and effective for that pur- 
pose, penicillin was coming out of the industry's 
ears. Had that NDA been looked at on the basis 
of the product's cost against its possible benefits 
in prolonging blood levels of penicillin, work on 
the drug would have been stopped in the early 
‘50s. Of course, once it was found that it speeded 
the excretion of uric acid, an entirely new and a 
relatively important means of relieving gout be- 
came a possibility. It isn’t simply a matter of 
NDA approval. If we knew we would have to 
pass that kind of test at the NDA stage, I fear 
that a lot of projects would be cancelled long be- 
fore we had practical information on the drug’s 
full potential. 


In recent years, however, FDA has approved 
very few NDAs to add to a previously approved 
product's official labeling a new major indica- 
tion that required data from new clinical trials. 
Between January 1, 1974, and September 30, 
1979, for example, FDA approved 4 such NDAs 
out of a total of 484 total NDA approvals for all 
reasons (221). There are, however, 362 active 
commercial INDs for products being used for in- 
dications not currently listed in the products’ of- 
ficial labeling. In addition, for perhaps hun- 
dreds of approved products, FDA has permitted 
manufacturers, through the supplemental NDA 
process, to make minor modifications (i.e, those 
not requiring manufacturers to sponsor new 
clinical trials) in their products’ official indica- 
tions for use. The extent to which marketed 
drugs are used for unapproved indications is not 
known. Manufacturers often have no economic 
incentives to seek FDA approval for popular, 
unofficial clinical uses of their products. 


Second, the calculation of costs needed to 
assess the cost effectiveness of a new drug would 
require FDA to either: 1) use existing estimates 
of costs of treating illnesses using alternative 
forms of therapy, or 2) generate new data bases. 
Problems encountered in using existing data 
bases are discussed in a background paper of 


this assessment.?? At present, FDA has no intra- 
mural capability for generating new data re- 
garding the treatment costs for selected diseases. 
Conceivably, FDA could ask or require drug 
and medical device manufacturers’ to submit 
estimates of the costs of treatments provided 
during the clinical testing of a new drug. Mean- 
ingful estimates of treatment costs might be dif- 
ficult to calculate, however, because the prices 
charged for medical care are dynamic and can 
vary substantially among geographical regions. 


At the time FDA approves a drug or medical 
device, it does not know—nor can it influence— 
how much a manufacturer will charge for its 
newly approved product. FDA could ask a man- 
ufacturer to estimate a product's selling price. 
Apparently, however, this task would be diffi- 
cult for manufacturers to accomplish at the time 
of market approval. Further, the price of a drug 
or medical device will likely change over time, 
and cost-effectiveness information calculated at 
the time of market approval would likely 
change. 


Third, the effect on competition of using cost 
effectiveness as a criterion for market approval 
of drugs and medical devices is unknown. Po- 
tential implications vary substantially depend- 
ing on how FDA might use the cost-effectiveness 
criterion. 


If FDA assumed the responsibility for con- 
ducting CEAs, theoretically manufacturers 
would not be burdened with the expense of de- 
veloping the capability to conduct such anal- 
yses. Most research-based companies, however, 
would likely establish their own capabilities; at 
least two pharmaceutical firms currently per- 
form CEAs on some of their products. One com- 
pany, Merck Sharp and Dohme, developed a 
computerized cost-benefit model to illustrate the 
costs and benefits of its pneumococcal vaccine 
(464). Another pharmaceutical manufacturer, 
SmithKline and French, has extensively studied 
the costs and benefits of the use of one of its 
drugs, cimetidine, in different populations 
(529). If FDA required each sponsor of every 


22Background Paper #1: Methodological Issues and Literature 
Review, prepared by OTA. 


new drug or medical device to conduct CEA as a 
part of the premarketing approval processes for 
drugs and devices, then the use of cost effec- 
tiveness would add to the financial and regula- 
tory hurdles of new product introduction. Con- 
ceivably, that situation could reduce manufac- 
turers’ willingness to bring a new product to 
market and could reduce competition. 


If FDA required only the original manufac- 
turer of a new drug or medical device (i.e., one 
not previously marketed in the United States) to 
analyze the cost effectiveness of its product, 
then the burden of conducting such analyses 
would fall primarily on the leading innovating 
companies. 


If FDA used cost effectiveness as a criterion 
for the marketing of only new therapeutic enti- 
ties, and not for either generic or ‘‘me too” prod- 
ucts,2> and if manufacturers perceived such a 
task as too expensive, then manufacturers might 
attempt to break into existing markets for multi- 
ple source products rather than to develop new 
drugs and create new markets. 


If FDA applied cost-effectiveness criteria to 
“me too” type products, then a manufacturer 
might lower the introductory price of its new 
product in order to make it compare favorably 
to already-approved products in the same thera- 
peutic category. Once a product was marketed, 
however, its manufacturer could alter its price, 
and FDA has no authority to control the price of 
approved drugs and medical devices. Excessive 
increases in postapproval prices would likely be 
limited by competition in the marketplace for 
similar products. 


Fourth, the use of CEA to evaluate new drugs 
and medical devices would require extensive re- 
sources, substantial time, and creative applica- 
tion of existing data. FDA would likely have to 
compare the relative safety, efficacy, and cost of 
an applicant product with those same character- 
istics of marketed products. In order to perform 
such comparisons, FDA would have to do the 
following: 


23"Me too” products are those that are chemically dissimilar 
enough to approved products to warrant a separate NDA review, 
but that nevertheless produce quite similar net therapeutic effects. 
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© quantitatively assess the safety (risks) and 
efficacy (benefits) of each marketed drug 
and medical device; 

e establish standards for clinical efficacy, 
safety, and cost of each available form of 
treatment—and possibly prevention—in 
numerous disease states (note: such stand- 
ards could be incorporated into a mono- 
graph system such as those used for OTC 
drug products and antibiotics); and 

© calculate cost-effectiveness information for 
the use of each drug and medical device in 
specialized populations, such as the elderly 
and persons with specific medical prob- 
lems. 


Although it is debatable whether FDA has 
statutory authority to evaluate new products 
relative to currently marketed products, a pro- 
vision in the Senate-passed version of the Drug 
Regulation Reform Act of 1979 would allow 
FDA to consider formally and explicitly the 
“benefits and risks of available therapies” when 
evaluating applicant drugs in the NDA review 
process. 


Fifth, the cost effectiveness of different prod- 
ucts would vary substantially depending on the 
forms of treatment being compared. A compari- 
son between a new drug and a surgical proce- 
dure in the treatment of a selected medical prob- 
lem, for example, could yield large differences 
in cost-effectiveness ratios. Such a comparison 
might be useful to FDA in its evaluation of the 
drug. The information yielded could be helpful 
in the evaluation of the new product, especially 
if the product were a new chemical entity that 
represented a therapeutic breakthrough. Ulti- 
mate treatment for uncontrollable malignant 
hypertension in a young person, for example, 
has been the removal of one or both kidneys 
(nephrectomy). Until recently, many cases of 
this disease have not been controllable through 
the use of drugs. A potent new antihypertensive 
drug, minoxidil, has been shown to effectively 
lower very high blood pressure, and in some 
studies, its use reduced the need for nephrec- 
tomies (482). There is no other product on the 
market that resembles minoxidil either chemi- 
cally or therapeutically. When compared to sur- 
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gical removal of the kidneys, minoxidil would 
likely be quite cost effective for some patients. 


Comparisons between two drugs, however, 
especially two drugs with similar therapeutic ef- 
fects, would likely yield only small differences 
in cost-effectiveness ratios. Small differences 
would be of little value. Thiazide diuretics, for 
example, represent another form of antihyper- 
tensive drug therapy. There are approximately 
30 different single-entity thiazide diuretic prod- 
ucts on the market, and each one produces very 
similar therapeutic and adverse effects. In cost- 
effectiveness comparisons among these 30 prod- 
ucts, the differences in cost-effectiveness ratios 
would likely be very small; hence such compari- 
sons would likely be of little distinguishing val- 
ue in FDA's NDA review process for a new thia- 
zide diuretic. 


Sixth, the cost effectiveness of a drug or a 
medical device might be influenced more by the 
conditions of its use than by its demonstrated ef- 
ficiency in premarketing clinical trials. Factors 
such as dosage regimen, route of administra- 
tion, and palatability (taste) all could influence 
a product's acceptance by patients and hence af- 
fect its cost effectiveness. Even if FDA were able 
to accurately assess the cost effectiveness of a 
drug or medical device at the time of market ap- 
proval, such an assessment might not accurately 
predict the efficiency of the product in general 
use. 


Seventh, if FDA used cost effectiveness as a 
criterion in its drug and medical device market 


approval processes, the Agency would have to 
rely on speculative and uncertain data for two 
important valuations. First, FDA measures the 
safety and efficacy of drugs and medical devices 
in terms of changes in physiologic functions, not 
in terms of changes in a person’s overall health 
status. The efficacy of an antihypertensive drug, 
for example, is evaluated on the degree to which 
the drug reduces a person's blood pressure. FDA 
does not quantify the effect a drop in blood 
pressure would likely produce in the health 
status of a hypertensive patient. Second, FDA 
analysts would face the challenge of assessing 
the economic value of physiological changes— 
and if possible of those in health status—among 
persons using the medical device or drug. 


Eighth, by using cost effectiveness as a cri- 
terion in the market approval process for drugs 
and devices, FDA might be extending its role 
beyond the scope of responsibilities Congress 
intended the Agency to have. FDA's primary 
purpose is to protect the public from unsafe and 
ineffective drugs and medical devices and from 
unsafe cosmetics, foods, and food additives. 
Thus far, Congress has not asked FDA to use 
economic criteria to regulate the choice of safe 
and effective products during the market ap- 
proval process. Choices based on economic cri- 
teria are left to be made by consumers, health 
care practitioners, hospital administrators, and 
private and public health insurance carriers 
after a product has been marketed. 
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9. 
R&D Programs 





INTRODUCTION AND BACKGROUND 


The Federal Government supports’ a wide 
range of health R&D activities. The object of 
this research, no matter what its form, is the 
production of knowledge. Some research is de- 
signed to yield information on health itself, or 
on diseases and disabling conditions. Some is 
designed to produce new tools, technologies, to 
intervene in disease processes, or to counteract 
the effects of disease. Some research evaluates 
those tools; other research investigates the use 
of technology and other aspects of the health 
care delivery systems. Much of the existing re- 
search serves multiple purposes, and some 
yields results that are more valuable to other 
fields than to the field the research was designed 
to address. 


R&D have given the health care system and 
this country a great deal of beneficial informa- 
tion and many effective technologies, but are ac- 
tivities fraught with uncertainties. These ac- 
tivities can also be expensive ones—billions of 
dollars are spent on health R&D each year in 
this country by the Federal Government alone. 
The expenditure of Federal funds for R&D is an 
investment in the future—and much of this in- 
vestment represents a public good. Therefore, it 
is important that moneys for health R&D be 
spent as wisely as possible and in accord with a 
balance between public and scientific priorities. 


R&D support by Federal agencies can be 
viewed from two perspectives: process and sub- 
stantive. The former involves questions of what 
methods are used to make administrative deci- 
sions regarding selection of priorities, program 
direction, or projects to be funded within each 
agency. The issues relate to selection and pur- 
suit of goals and to the methods employed to as- 
sist in efforts to allocate resources. The latter 
perspective, “substantive” performance, in- 
volves questions and issues pertaining to the 
agencies’ performance or sponsorship of re- 
search that examines the allocation of techno- 


logical or other resources in the health care sys- 
tem. 


Since this OTA assessment focuses on the use 
of cost-effectiveness analysis/cost-benefit anal- 
ysis (CEA/CBA), the chapter concentrates on 
questions dealing with that analytical tech- 
nique. Four Federal agencies’ activities in regard 
to CEA/CBA from both of the aforementioned 
perspectives are examined. If an agency does 
not support or use CEA/CBA, the methods and 
procedures it does use are discussed briefly. Al- 
though the distinction between the process and 
substantive perspectives of research support is 
at times unclear, an attempt is made in the dis- 
cussion below to separate them as clearly as 
possible and to note the similarities when the 
situation requires. 


Research Classifications.—Health R&D is an 
umbrella term used to describe a wide range and 
diversity of activities. A single scientist focusing 
on an extremely narrow topic within the field of 
biomedical science is doing health research. The 
same can be said of the analysts that are examin- 
ing the performance of a patient screening pro- 
gram, developing physician productivity meas- 
ures, or working on any number of other health 
care issues. 


To help describe the primary focus of health- 
care-related R&D, the range of activities is often 
broken into four loosely defined categories: 
1) basic research, 2) applied research, 3) appli- 
cation (or transfer) research, and 4) health serv- 
ices research. The demarcations between these 
four research categories are not clearly defined. 
Nevertheless, the classifications play an impor- 
tant role in the process of setting health care re- 
search priorities, allocating and distributing 
funds, and evaluating outcomes or products of 
R&D efforts. At several levels in the health care 
decisionmaking and policy process, the in- 
tended purpose of a given research effort is im- 
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portant. There is constant tension in the deci- 
sionmaking process between those who advo- 
cate the allocation of funds for increased sup- 
port of basic research, those who feel more 
work is needed in applying more fully the 
knowledge and technologies that exist (applica- 
tion or transfer research), and those who cite a 
pressing need to examine what is already in 
place and how well it is working or how to 
make it work better before adding more to the 
system (services research). As a result of these 
different perceived research needs, the research 
“label” that is affixed to a given health care pro- 
gram or initiative can be quite important to its 
ultimate success in the resource allocation proc- 
ess. 


For purposes of the discussion below, the Fed- 
eral health care R&D effort is divided into two 
broad general categories: 1) biomedical re- 
search, and 2) health services research. Biomedi- 
cal research includes basic, applied, and, to a 
degree, application or transfer research. Health 
services research includes work done on tech- 
nologies or systems that are still considered to 


BIOMEDICAL RESEARCH—NIH 


NIH is not the only Federal agency to conduct 
or fund biomedical research. The Veterans Ad- 
ministration, the Office of Naval Research, the 
National Science Foundation (NSF), the Depart- 
ment of Energy, and others are involved to 
varying degrees in a range of biomedical re- 
search activities. NIH, however, is by far the 
largest single provider of biomedical research 
funds in the United States. NIH covers a wide 
range of scientific activity and uses peer review, 
as well as program and project evaluation, proc- 
esses that are similar to those of other Federal 
agencies that support health care R&D. 


NIH is an agency of the Public Health Service 
(PHS) in the Department of Health and Human 
Services (DHHS). Its mandate, stated in broad 
terms, is to improve human health by increasing 
understanding of the processes underlying 
health and acquiring new knowledge to prevent, 
detect, diagnose, and treat disease and disabili- 
ty. 


be in the development, transfer, and application 
stages, as well as on technologies or systems 
that are in widespread use. Actually, the lines of 
definition in terms of where biomedical research 
activities end and where health services research 
begins are rather blurred. This is because it is 
rare for a technology or innovation to proceed 
in a linear process from basic research to wide- 
spread application. It is also unlikely that a 
single agency or other institution will have an 
innovation under its purview for the full range 
of developmental needs that are part of produc- 
ing a usable end product. 


In the discussion that follows, the National 
Institutes of Health (NIH) is used to represent 
the biomedical research process. The National 
Center for Health Services Research (NCHSR) 
and the Health Care Financing Administration 
(HCFA) are the examples of Federal agencies 
that sponsor health services research. And the 
National Center for Health Care Technology 
(NCHCT) is the example of an agency whose ac- 
tivities incorporate or relate to both types of 
R&D. 


NIH pursues this mission via an array of 
intramural programs conducted at NIH and 
through an extensive network of extramural 
grants and contracts to private and public insti- 
tutions in the United States and other countries. 
Its budget in 1980 will be approximately $3.4 
billion, which represents approximately 68 per- 
cent of the Federal obligations for health R&D. 
Forty-one percent of total national health R&D 
support (Federal and State Government, indus- 
try, and private nonprofit organizations), is 
provided by NIH (1978 estimates) (446). 


NIH is organized into 11 institutes, the Na- 
tional Library of Medicine (NLM), and 6 re- 
search and support divisions. Two of the insti-- 
tutes (the National Cancer Institute (NCI) and 
the National Heart, Lung, and Blood Institute 
(NHLBI)), as well as NLM, have “bureau” sta- 
tus; the other institutes are “division” level orga- 
nizations. These various semiautonomous orga- 


nizations are coordinated through the Office of 
the Director of NIH. 


There are several levels of control and input 
in the resource allocation process for biomedical 
research.1 Every year, NIH is subjected to nu- 
merous examinations of its allocation of re- 
search support, its selection and implementation 
of research priorities, and its requests for funds 
for the upcoming year(s). This process extends 
from congressional hearings on NIH budget au- 
thorization and appropriation levels to the 
study section advisory groups that meet three 
times each year to evaluate the technical and/or 
scientific merit of research proposals. Further- 
more, it is not unusual for an ad hoc presidential 
or congressional panel, commission, or task 
force to express its opinions and conclusions re- 
garding the quality, quantity, or usefulness of 
NIH’s efforts. These “advice-giving” groups can 
carry considerable weight in the priority-setting 
and allocation process. Another group that can 
affect the amount of support given a particular 
research area are the scientists themselves. The 
thousands of scientists who continue to initiate 
and support various types of research, submit 
grant and contract proposals, and remain in an 
area of research for extended periods of time can 
have considerable bearing on national research 
priorities. 


Congress plays, and has played, a significant 
role in the creation, expansion, and contraction 
of research efforts at NIH. Its budget-setting and 
oversight authority are powerful levers in the 
decisionmaking and_ allocational process. 
Strickland (586,587), Ward (612), and others 
(91,367) have noted that medical research is a 
national policy issue that entails all the political 
pushing and shoving that is characteristic of 
other national allocational issues. The alloca- 
tion of funds for medical research is neither just 
a budgeting exercise nor a purely scientific deci- 
sion that is, or will be, made only by the scien- 
tific community. In.part, this is because medical 


INIH is frequently praised for its contribution to the fields of sci- 
ence and medicine, and its achievements in these areas have been 
described as extraordinary. Yet in recent years, the charge has 
been leveled that its accomplishments have not been great enough 
given the large sums of money that have been channeled its way. It 
is beyond the intent or capabilities of this study to enter this de- 
bate. 
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research, technological advances, new treat- 
ments and cures, and the health care research 
system itself have come under the scrutiny of the 
general public. Reader's Digest, Ann Landers,? 
Marcus Welby, M.D., disease-oriented interest 
groups, and others have turned millions of 
Americans into supporters or critics of various 
aspects of biomedical research, in particular, 
and the health care system, in general. 


Congress is in the position, both constitution- 
ally and politically, to exercise a great deal of 
control over the priority-setting and allocation 
process at NIH through its ‘power of the purse.” 
Maldonado sums up congressional influence via 
“the appropriating art” (379): 


Health budget review and development fall 
under the jurisdiction of the subcommittees on 
Labor-HEW (of the House and Senate commit- 
tees on appropriations). The subcommittee on 
Labor-HEW has authority* to (1) recommend 
the appropriations or funding level in support of 
program or research activities (or their termina- 
tion); (2) through the report vehicle, earmark 
funds for specific programs or projects, set pro- 
gram directions, instruct, warn, and exhort; (3) 
accept or reject proposed impoundments (rescis- 
sions and deferrals); (4) approve or disapprove 
transfer of unexpended balances; and (5) con- 
duct studies and examinations of agency/de- 
partment operations and organizations. 


The appropriations subcommittees have tradi- 
tionally played a prominent role in health policy 
by “earmarking” funds for specific activities, 
and “requesting” or “expecting” that a certain 
“emphasis” or direction be taken.** In sum- 
mary, they set program direction, emphasis, and 
budget levels for health. 


A number of other forces operate both within 
and outside Government to ensure that a given 
health care issue receives attention in the deci- 


2An example of the tremendous response that can be generated 
by “popular” authors, TV shows, publications, etc., is given by 
Robert Q. Marston regarding the debate between those who did 
not want a “cancer crusade,” and those who did. He cites the fact 
Ann Landers stimulated more than a million responses to a state- 
ment in one of her columns in 1971 asking for support of the can- 
cer crusade (382). 

*W. H. Brown, Rules of the House of Representatives, pp. 
349-351, 393, 396-399, Washington, D.C.: U.S. Government 
Printing Office, 1975 (71). 

**R. F. Fenno, The Power of the Purse: Appropriations Politics 
in Congress, Little, Brown, Boston, 1966. (199). 
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sionmaking process. The executive branch can. 


marshall a considerable collection of expertise— 
from NIH, the Food and Drug Administration 
(FDA), HCFA, special task forces of experts, 
and numerous advisory councils that are part of 
the R&D process—to provide advice to the pol- 
icy process. The scientific community, the many 
disease-oriented organizations (e.g., the Ameri- 
can Cancer Society, the Cystic Fibrosis Founda- 
tion) and professional organizations (e.g., the 
American Medical Association, the American 
Hospital Association) provide a mix of voices 
that add to the diversity of views on various 
health care issues. At any time, on any given 
health care issue, there are likely to be coalitions 
within both the legislative and executive 
branches of Government that have the support 
or opposition of the many nongovernmental in- 
terest groups—all urging special consideration 
for their programs on the research agenda. For a 
detailed description of this process, see Rettig’s 
(510) and Strickland’s (587) accounts of the 
“war on cancer” declared in 1971 by President 
Nixon. 


Rarely are formal decision-assisting tech- 
niques, especially CEA/CBA, explicitly used to 
make decisions at this broad political or societal 
level of the policy process. Although the preced- 
ing discussion has been based on NIH, much of 
what was said about decisions at this policy lev- 
el pertains to the health services research sys- 
tem, as well. 


Peer Review Allocation and 
Evaluation Mechanisms 


NIH and each institute within NIH must de- 
cide how to divide available resources among: 
1) extramural grants, 2) contract research, and 
3) intramural projects initiated by scientists 
within NIH. In the case of extramural grants, 
further consideration must be given to the 
allocation of resources for investigator-initiated 
research grants and large, complex multidisci- 
plinary research team efforts such as center and 
program project grants. To some extent, NIH 
priority-setting and research selection is based 
on the relative merits of basic, applied, or trans- 
fer research in each institute’s area and the 
budget the institute will receive. 


The mechanism evolved at NIH to manage 
these many considerations is a peer review sys- 
tem. Most, if not all, of the Federal agencies that 
support health care research rely on some form 
of peer review to solicit expert opinion regard- 
ing the potential success of a proposed project. 
The peer review system of NIH consists of 2,200 
primarily non-Federal scientists and lay advi- 
sors from across the Nation. These individuals 
are grouped into 130 peer review groups, advi- 
sory committees, councils, and panels (449), 
whose function is to provide NIH with expert 
opinions both on the scientific and technical 
merit of grant applications and contract propos- 
als and also on program initiatives and policy 
issues. 


Extramural Grants 


The peer review system for grant applications 
used by NIH is based on two sequential levels of 
review, referred to as the “dual review system.” 
The first level involves panels of experts estab- 
lished according to scientific disciplines or cur- 
rent research areas for the primary purpose of 
evaluating the scientific and technical merit of 
grant applications. In the Division of Research 
Grants (DRG), discussed below, the panels are 
called study sections. Generally, however, the 
panels are referred to as initial review groups 


(IRGs). 


The second level of review is by a national ad- 
visory council or board, referred to here as a 
“council.’” Council recommendations are based 
not only on considerations of scientific merit as 
judged by IRGs, but also on the relevance of a 
grant application to an institute’s programs and 
priorities. 


Receipt and Assignment of Applications. — 
Grant applications submitted to NIH are re- 
ceived centrally in DRG. This Division, one of 
the research and support divisions at NIH, is not 
connected to the research institutes or responsi- 
ble for funding or managing grant programs. 


DRG screens incoming grant applications, de- 
termines the relevance of each application to the 
overall mission of NIH, and assigns acceptable 
applications to an appropriate IRG and to an 
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appropriate institute.* Assignment to an IRG is 
based on the complementarity of a proposed re- 
search project to the review responsibilities and 
scientific expertise of IRG’s members; assign- 
ment to an institute is based on the institute's 
legislatively mandated program responsibility. 
If the subject matter of an application is perti- 
nent to the program responsibilities of two insti- 
tutes, a dual assignment may be made. Should 
the primary institute decide not to provide fund- 
ing, the other institute may consider the applica- 
tion for funding. 


Initial Review by Peers.—Depending on the 
type of research proposed, the first level of sci- 
entific and technical merit review is by an IRG 
located either within DRG or within an insti- 
tute. IRGs in the institutes are usually multidis- 
ciplinary and are thus constituted to review 
more complex program project and center grant 
applications. An NIH health scientist adminis- 
trator serves as executive secretary of each re- 
view group. 


IRG members, who serve up to 4 years per 
appointment, meet three times a year to review 
applications. When assessing the scientific and 
technical merit of an application assigned to 
their IRG, the members consider, among other 
criteria: the importance of the proposed re- 
search problems; the originality of the ap- 
proach; the training, experience, and research 
competence or promise of the investigators; the 
adequacy of the experimental design; the suit- 
ability of the facilities; and the appropriateness 
of the requested budget to the work proposed 
(164). 


For each application, IRG makes a recom- 
mendation of approval, disapproval, or deferral 
for additional information by majority vote. In 
addition, for each application recommended for 
approval, each member of IRG individually and 
privately records a numerical rating that reflects 
a personal evaluation of the scientific merit of 
the proposed research or training. The numer- 
ical rating is from 1.0 (the most meritorious) to 


3The Referral Branch also receives and assigns applications to 
other agencies within PHS, such as the Alcohol, Drug Abuse, and 
Mental Health Administration, the Center for Disease Control, the 
Health Services Administration, the Health Resources Administra- 
tion, and the Food and Drug Administration. 


66-220 0 - 80 - 8 


5.0 (the least meritorious), with 0.1 increments. 
After the meeting, the executive secretary aver- 
ages the individual reviewers’ ratings for each 
approved application and multiplies this by 100 
to provide a three-digit number known as the 
priority score. Priority scores assist the staff of 
the institutes in determining which applications 
are to be funded. 


If information is needed that is not in the ap- 
plication and cannot be obtained by telephone 
or mail, a project site visit may be made either 
prior to an IRG meeting or after an IRG deferral 
recommendation. In addition, site visits are 
often routinely made when an application in- 
volves complex coordination. In either case, the 
executive secretary assembles a team of site 
visitors. For a research project grant applica- 
tion, the site visit team generally includes two or 
more members of IRG, the executive secretary, 
a member of the institute staff, and usually one 
or more ad hoc consultants who are experts in 
critical aspects of the proposed work. For more 
complex grant applications (e.g., those for pro- 
gram project or center grants), the site visit team 
may include as many as 10 to 15 individuals, in- 
cluding members of IRG and ad hoc consult- 
ants. 


After IRG meets, the executive secretary pre- 
pares a summary statement for each application 
and forwards it to the appropriate institute for 
review by its council. The summary statement 
contains a description and critique of the pro- 
posed activity, an explanation of the recommen- 
dation of IRG, a recommended budget, and no- 
tations about any special points (e.g., a split 
vote or a potentially hazardous experimental 
procedure). 


Review by National Advisory Councils or 
Boards.—Each NIH funding unit has a council 
that must review and recommend action on the 
applications received from IRGs. These councils 
are mandated by law, and some have minimum 
levels placed both on the number of times they 
must meet each year and on the number of. 
members they must have.* Members include au- 


‘For example, the National Cancer Advisory Board was estab- 
lished in August of 1937 by Public Law 244, 75th Cong., and has. 
since been renamed and restructured by subsequent legislation that 
required that the board shall meet “not less than four times a year” 
(449). 
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thorities in scientific and health fields directly 
related to the program interests of the institute, 
as well as lay people noted for their interest or 
activity in national health problems. With the 
exception of individual fellowship applications 
and some grant applications recommended at 
yearly levels not exceeding $35,000 (in NCI and 
NHLBI), grants cannot be awarded without ap- 
proval by a council. 


The councils review research proposals in a 
broader context than IRGs, because the coun- 
cils’ recommendations are based not only on 
IRG scientific and technical merit evaluations, 
but also on the needs of NIH and the missions of 
the individual institutes, the need for initiation 
of research in new areas, the degree of relevance 
of the proposed research to the missions of the 
institutes, and other policy issues (164). In addi- 
tion, a major focus of the councils is the SATT® 
criteria that have been developed to guide and 
categorize the support for biomedical research. 


The councils forward their recommendations 
for each application to the institute director for 
a decision on funding. Usually, the approved 
projects are chosen according to rank until the 
budget is obligated. An approved grant applica- 
tion is not assured of funding, because there are 
almost always more eligible applications than 
available funds. An unapproved application, 
however, cannot be funded (164). 


R&D Contracts 


Contracts are used to procure a specified re- 
search product or service from a nonprofit or a 
commercial organization. The initiative for 
these contracts generally comes from NIH pro- 
gram staff and advisors, who identify a specific 
research or service need. Workshops and con- 
ferences are a source of invaluable ideas and 
guidance. 


°S stands for Science Base activities. This category is, for all 
practical purposes, basic research support. A is clinical Appli- 
cation, research focused on intervention. It is at this level that clin- 
ical trials take place. T represents the Transfer research that NIH 
undertakes to move products or interventions that have survived 
testing to the consumers and providers of health care. Demonstra- 
tions, practical trials, and consensus development conferences are 
part of the transfer process. The final T stands for the Training 
function that NIH supports. This effort is geared to supporting and 
attracting people into the field of research. 


Each institute has developed slightly different 
methods and procedures for using contracts to 
satisfy its research needs. The basic mechanisms 
the various institutes use to develop contract 
proposals, to review contract applications, and 
to evaluate the progress and outcomes of con- 
tract products, though, are similar enough to be 
summarized in a general description. 


The scientific staff members within a given in- 
stitute, with assistance from standing commit- 
tees or ad hoc advisory groups, develop a re- 
search project description and plan. In compli- 
ance with the law that mandates peer review for 
NIH contract projects, the concept of the project 
is evaluated by a scientific review group com- 
posed largely of non-Federal advisors. The pro- 
posed project is then released as an RFP (request 
for proposal), which specifies the terms, condi- 
tions, and provisions for the requested contract. 
RFPs are announced in the Commerce Business 
Daily as required by law, in the NIH Guide for 
Grants and Contracts as required by policy, and 
in other appropriate publications as determined 
by program requirements. 


In response to an RFP, applicants submit con- 
tract proposals, which are reviewed by the in- 
stitute’s contracting officer and then by a scien- 
tific review group consisting mainly of non- 
Government scientists chosen for their expertise 
in the relevant area. Their recommendations are 
sent to a contract review committee composed 
of senior program staff from the funding insti- 
tute. Applicants determined to be in the “com- 
petitive range” have an opportunity to further 
defend or clarify their proposals via written or 
oral discussion with the contracting officer and 
senior program staff. Once the applicants have 
made their “best and final’ offer, the remaining 
applications are reevaluated via further negotia- 
tions in order to determine the one to be funded. 


In addition to contracts solicited by NIH, un- 
solicited contract proposals are also occasional- 
ly received by DRG, and then sent to the appro- 
priate institutes. If the unsolicited proposal is 
relevant to the institute’s program needs, it is re- 
viewed by the contracting officer and scientific 
review group in a process similar to that for so- 
licited proposals. 
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The progress and products of contract re- 
search are under the supervision and review of a 
project officer at the funding unit. Informal, as 
well as formal, procedures are used to monitor 
the performance of the contract project. A ma- 
jor difference between contract research and 
grant and intramural research, at least theoreti- 
cally, is that contractors are required to provide 
an end product based on specifications estab- 
lished by the funding unit before the research 
begins. The other forms of research support are 
usually not as tightly bound by requirements to 
produce a given outcome at the end of their re- 
search. 


Intramural Research 


Intramural research projects are developed, 
supported, and evaluated by a mixture of insti- 
tute staff and outside advisors. Each institute 
has an in-house scientific board that meets on a 
regular basis to set institute policy, review in- 
stitute programs, and discuss research needs. 
For each institute, the scientific board also iden- 
tifies future goals, needs, and capabilities for in- 
tramural research. 


Intramural research ideas or project proposals 
can be initiated by the scientific board, by indi- 
vidual researchers, laboratory chiefs, or by sci- 
entific directors. Research proposals are dis- 
cussed by in-house scientific staff, the institute's 
scientific board, and outside experts if needed. 
Depending on the available space, personnel, 
and budget, and the feedback from this informal 
peer review process conducted in-house, a spe- 
cific project is started, rejected, or deferred. 


Additional aspects of the system are publica- 
tion of research in journals and presentation of 
work to in-house staff and the scientific commu- 
nity. Directors from each institute meet each 
month to review the work of selected nonten- 
ured researchers who are candidates for tenured 
status at NIH. Finally, each institute has a Board 
of Scientific Counselors, composed of non-Gov- 
ernment scientists, that meets twice each year to 
review intramural projects and programs for 
scientific performance and progress. 


Discussion of Peer Review Mechanisms 


The peer review mechanisms at NIH, by and 
large, appear to have worked quite well over the 
last several decades. This is not to say that the 
peer review system has been beyond criticism or 
change. There have been a score of studies, 
hearings, reports, and reviews on the peer re- 
view process at NIH over the last 25 years, and 
these many assessments have led to reevalua- . 
tions and modifications of the procedures used 
to conduct the research support processes.® De- 
spite these changes, however, the fundamental 
concept and framework of the peer review sys- 
tem at NIH remain intact. Furthermore, the peer 
review system remains the mechanism of choice 
for efficiently and effectively allocating research 
resources. 


The research and resource allocation process, 
both in biomedical and health services research, 
is an uncertain endeavor that entails probability 
and risk. The peer review process is an attempt 
at predicting the probability of success for a 
given scientific effort. Through this process, 
NIH and the other research-supporting agencies 
attempt to estimate or predict a subjective level 
of quality performance of a researcher, given 
the very real confines of a budget, a limited time 
frame, the existing knowledge and technological 
base, the availability of trained researchers will- 
ing and able to work on a given problem, re- 
search space and resources, and the presence of 
often conflicting and changing health care goals 
and policies. In a very real sense, the peer re- 
view process is an attempt, either explicit or im- 
plied, to select and support “cost-effective” re- 
search. 


*Since 1947, there have been approximately 18 official reviews, 
studies, inquiries, hearings, and reports examining the adequacy of 
the peer review system at NIH (447). The most recent study to 
assess the mettle of the peer review system was supervised by Dr. 
Ruth L. Kirchstein, Director of the National Institute of General 
Medical Sciences. The study team received its go ahead in April of 
1975 and submitted phase I of the report to the Director of NIH in 
December of 1976, and the final report, phase II, in December of 
1978 (445,447,448). For additional views on the peer review sys- 
tem, see Report of the President's Biomedical Research Panel (491); 
Opinions on the NIH Grants Peer Review System, Phase II of the 
Report to the Director (447); Investigation of the National Insti- 
tutes of Health (115); and “Support of New Principal Investigators 
by NIH: 1966-1972” (153). 
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Performance Evaluations 


Even though NIH very rarely uses or funds 
CEA/CBAs in either the allocation or evalua- 
tion process (see survey of CEA/CBA use at 
NIH in app. B), the various institutes do employ 
a number of analytical techniques to assess the 
performance of the research process. Most of 
the techniques are employed after a “product” 
has been turned in. It is important to note, how- 
ever, that allocation and evaluation efforts at 
NIH often merge into the same function. In the 
minds of most NIH administrators and program 
and project leaders, evaluation activities are 
ongoing efforts that cannot be clearly or neatly 
separated into the categories of planning, allo- 
cation, and oversight. A number of the tech- 
niques that NIH, other research-supporting 
agencies, and analysts in general use to evaluate 
the performance of research efforts are de- 
scribed in the following paragraphs. 


General Asessments of Biomedical Research 


Citation Analysis.—A form of performance 
evaluation that has gained considerable atten- 
tion over the last decade is the use of citation 
analysis. This analytical technique examines the 
quantity and, to a degree, the quality of scien- 
tific papers, reports, articles, and so on, that re- 
sult from research projects. NIH, NSF, and 
others have used this technique to evaluate the 
performance of their research efforts. NIH and 
NSF have supported a number of studies over 
the last 5 years that tested: 1) the correlation be- 
tween their support efforts and biomedical pub- 
lication output (225,425); 2) the extramural role 
of NIH as a research support agency (426); 3) 
the relationship between the peer review system, 
citations, and biomedical research policy (86); 
and 4) other aspects of research support (565). 
Comroe and Dripps (120) and the Department 
of Defense (565) have employed a variation of 
citation analysis to evaluate the development of 
specific scientific and technological innovations. 


Morbidity/Mortality Studies.—Another type 
of performance evaluation is the attempt to 
measure the reductions in mortality and/or 
morbidity that have occurred as a result of bio- 
medical research and health care in general. 
Morbidity/mortality studies attempt to examine 


the achievements of biomedical research and the 
health care system that are responsible for pro- 
longing life, improving quality of life and avert- 
ing health care costs, as well as a host of other 
averted costs and added benefits that have re- 
sulted from the investment in biomedical R&D 
(9,421,422,512). 


In essence, both techniques above examine 
the usefulness of a field of science over the 
course of a number of years. For the immediate 
needs of institute, program, or project manag- 
ers, more specific decision-assisting techniques 
are required. 


Evaluations of Specific Technologies 
and Programs/Projects 


To assess the performance of specific medical 
technologies and programs and projects that it 
administers, NIH rarely uses explicit cost-effec- 
tiveness or cost-benefit studies, but it does use a 
variety of effectiveness studies. For this report, 
the effectiveness studies will be grouped into 
two categories: 1) clinical trials and consensus 
development conferences, which are used to 
evaluate the safety and efficacy of specific tech- 
nologies; and 2) more general performance eval- 
uations, which are used to examine a diversity 
of NIH programs and projects. Whereas clinical 
trials deal primarily with medical technologies 
that are in the development/application stage of 
research, performance evaluations can cover a 
myriad of types of NIH activity ranging from 
biomedical research to health services research. 
These evaluations are used for obtaining infor- 
mation necessary for program and administra- 
tive needs. 


Clinical Trials and Consensus Development 
Conferences.—Clinical trials provide the basis 
for the testing, evaluation, and application of 
basic and applied research knowledge before it 
is introduced into the health care system. They 
also provide the information needed to examine 
the safety and efficacy of newly emerging tech- 
nologies. NIH is the major source of support for 
these trials in the Federal Government (465). 
Clinical trials are technical in nature and are not 
usually designed to examine in depth any fac- 
tors other than the safety and efficacy of a medi- 
cal technology. 


Augmenting the clinical trial process are the 
recently implemented consensus development 
conferences that NIH sponsors. These confer- 
ences are designed to go a step beyond the lim- 
ited focus of clinical trials to assess a broader set 
of issues. A sample of the issues discussed at re- 
cent meetings are methods of diagnosing and 
treating allergies, treatments of ocular mela- 
noma, and estrogen use in postmenopausal 
women. These meetings may examine the scien- 
tific merit, along with the attending issues of 
risks, benefits, costs, and ethical implications, 
of implementing a new medical technology. The 
meetings are primarily a technically oriented ap- 
proach to discussing whether the innovation is 
safe, efficacious, and cost effective.’ 


Project Assessments.—The second type of 
evaluation is the much broader area of project 
assessment that NIH uses to examine completed 
and ongoing projects. All 11 institutes, NLM, 
and the Division of Research Resources support 
evaluation efforts within their areas of responsi- 
bility. 

Evaluation activities are divided into three 
general categories (450): 1) the NIH evaluation 
classifications, such as program effectiveness, 
methodology and resource development; 2) 
funding guidelines, i.e., science base, appli- 
cation, (technology) transfer, and training 
(SATT); and 3) the Assistant Secretary for Pro- 
gram Evaluation (ASPE) program evaluation 
categories. 


Four of the ASPE evaluation categories will 
be mentioned (450): 


e exploratory evaluations, which identify the 
objectives and expectations of relevant pol- 
icymakers and program managers, identify 
the program objectives and performance 
indicators on which the program will be 
held accountable, and identify evaluation/ 
management options for changing program 
activities, objectives, or uses of informa- 
tion in ways that are likely to improve pro- 
gram performance; 

e short-term evaluations, which summarize 
available and readily obtainable informa- 


7For a more detailed discussion of clinical trials and consensus 
development panels, see the OTA report Asessing the Efficacy and 
Safety of Medical Technologies (465). 
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tion on program performance in terms of 
the objectives and performance indicators 
identified in the exploratory evaluation and 
provide designs for full-scale evaluations; 

e full-scale evaluations of program perform- 
ance in terms of the agreed-upon measur- 
able objectives and performance indicators 
identified in the exploratory evaluations; 
and 

© program performance summaries, which 
summarize evidence on how programs are 
performing in terms of the set of objectives 
and performance indicators on which the 
program is being held accountable. 


The projects that fall into these four cate- 
gories, in that they attempt to examine explicit 
outputs and/or the inputs of a given project, are 
the closest NIH comes to conducting explicit 
cost-effectiveness studies. The boundaries estab- 
lished in the definitions above are not much dif- 
ferent from the general outlines used in many 
cases for CEAs. The studies’ main divergence 
from CEAs is in their relatively weaker empha- 
sis on “costs” and stronger focus on the per- 
formance, or effectiveness, aspects of the pro- 
gram or project. Cost comparisons remain im- 
plicit; the quality of project performance, how- 
ever, receives considerable attention. 


Potential Use of CEA/CBA 


The formal technique of CEA/CBA seems ill 
suited to the NIH biomedical research decision- 
making process (560). Perhaps this methodolo- 
gy could be helpful in certain program or proj- 
ect evaluation situations, or possibly in the con- 
text of assisting the awarding of contracts or 
supporting the research centers, but generally 
the complex, dynamic, and uncertain nature of 
the research process, the frailty of the methodol- 
ogy, and the backgrounds of the decisionmakers 
militate against the beneficial use of these tech- 
niques in the biomedical research system. 


The uncertainties involved in the biomedical 
R&D process are many, and CEA/CBA has lit- 
tle ability to adequately summarize, include, 
and compare items involved in bringing re- 
search to fruition. The development of a medi- 
cal technology does not follow a linear or steady 
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path. Biomedical science may contribute only a 
portion of the knowledge and research that is 
needed to develop an idea or technology fully. 
Even after the technology is in use, the obstacles 
to defining and measuring the costs, effective- 
ness, and outcomes are many. Distributional 
and equity issues present themselves at many 
points along the R&D process. These method- 
ological problems have yet to be solved. The list 
could be continued, but the heart of the issue is 
that formal CEA/CBA is not readily useful or 
applicable to the process of planning, allocat- 
ing, or evaluating biomedical research. 


The case against CEA and CBA grows even 
stronger when one examines the mechanisms 
already in place to assist the biomedical R&D 
decisionmaking process. Those mechanisms, the 
peer review system and evaluation processes, 
seem to have performed adequately over the 
years to allocate research resources efficiently 
and intelligently. As the rough edges are re- 
moved from those systems, they become even 
more valuable to the decisionmaking process. 
At the level of biomedical R&D, a cost-effective- 
ness attitude probably serves the system better 
than would formal CEA/CBA. 


NATIONAL CENTER FOR HEALTH CARE TECHNOLOGY 


In 1978, Congress added a new level of 
evaluation and coordination to the health care 
research, development, and application proc- 
ess by establishing NCHCT as part of HEW.§® 
NCHCT is responsible to the Assistant Secre- 
tary of Health, DHHS. Its mandate is to “under- 
take and support assessments of health care 
technologies.” 


Before the creation of NCHCT, there was no 
identifiable organization that could act as a 
coordinator for information concerning emerg- 
ing technologies. There was no single office that 
had the responsibility to act as the information 
manager for the application and dissemination 
of new medical technologies.° 


NCHCT has a potentially important role in 
the decisionmaking process. Its enabling legisla- 
tion establishes a number of broad-ranging 
functions for NCHCT. Generally, NCHCT is to 
set priorities for technology assessment and to 
encourage, conduct, and support assessments, 
research, demonstrations, and evaluations con- 
cerning health care technology. Specifically, the 
Center will (437): 


8The statute establishing NCHCT is Public Law 95-623, sec. 309 
of the Health Services Research, Health Statistics, and Health Care 
Technology Act, Nov. 9, 1978. 

°With respect to new medical technologies, the responsibility 
and involvement of NIH tends not to extend much past the applied 
research and early transfer stages. The health services research sec- 
tor usually does not focus on new technologies until they are in 
place or at least well along the development cycle. 


© undertake and support comprehensive as- 
sessments of health care technology, in- 
cluding analyses of safety and efficacy, and 
ethical issues; 

e undertake and support studies of the cost 
effectiveness and cost/benefit of current 
and developing technologies; 

e undertake and support syntheses of existing 
research (e.g., state-of-the-art papers); 

e provide the best scientific/medical and eco- 
nomic assessments to HCFA on medicare 
coverage for specific medical procedures 
and technologies, including evaluation of 
the costs and benefits of old procedures and 
assessment of new technologies for which 
HCFA might require medicare coverage de- 
cisions in the future; 

¢ undertake and support dissemination of in- 
formation derived from its assessment ac- 
tivities to the practicing and scientific com- 
munities, Federal agencies with health in- 
terests, third-party payers, the public, and 
others as appropriate; 

e undertake and support manpower training 
programs to provide for an expanded and 
continuing supply of individuals qualified 
to perform the research, demonstration, 
and evaluation activities related to health 
care technology; and 

¢ undertake and support, to the extent prac- 
ticable, by September 1, 1981, the plan- 
ning, establishment, and operation of three 
extramural centers for assessments, re- 
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search, demonstrations, and evaluations of 
issues in health. 


By law, NCHCT must have a national council 
to advise the NCHCT staff. A major function 
the council, which has been established, will 
serve is the identification and selection of med- 
ical technologies that should receive priority at- 
tention.’° The council is also asked to (142): 


¢ advise the Secretary on the safety, efficacy, 
effectiveness, cost effectiveness, and social, 
ethical, and economic implications of par- 
ticular health care technologies; 

e develop, publish, and disseminate stand- 
ards, norms, and criteria concerning the 
use of particular technologies, when appro- 
priate and practicable; and 

e review applications for grants and con- 
tracts exceeding $35,000 in direct costs. 


NCHCT is less than 2 years old. Much of its 
activity since its creation has been directed 
towards organizing and developing procedures 
to perform the functions it was created to serve. 
For that reason, it is difficult to examine 
NCHCT in the same light as NIH, NCHSR, and 
HCFA—agencies that, together, have several 
decades of experience behind them. 


Research Support Structure 


The research support structure of NCHCT is 
similar to that of NIH, NCHSR, and HCFA. 
NCHCT will support research via extramural 
grants, extramural contracts, intramural re- 
search, and will support manpower training 
programs to draw qualified individuals into this 
area of research. The national council is respon- 
sible for reviewing any grant exceeeding $35,000 
in direct costs, and outside experts can be used 
to review and provide comment on any applica- 
tions for research funds or any results of re- 
search that are received by NCHCT. The peer 
review system and the basic organizational pro- 


10The NCHCT National Council identified the following tech- 
nologies as candidates for priority consideration (457): 

Ultrasonic diagnostic procedures; coronary by-pass surgery; fetal 
monitoring; end-stage renal disease—dialysis (home v. hosp.), trans- 
plantation, etc.; non-invasive radioactive imaging; barium enema; 
EEG; nuclear magnetic resonance; auto-analyzers; all skull films; cere- 
bral angiography; dental X-rays; CAT scanner; continuous flow anal- 
ysis. 


cedures for selecting and supporting research at 
NCHCT are much the same as elsewhere. 


Technology Evaluation Activities 


NCHCT is in the process of pursuing a range 
of research and dissemination activities and is in 
the planning stages for several additional proj- 
ects to be initiated this year (1980). At the first 
national council meeting in October of 1979, 
Ruth Hanft, Acting Deputy Assistant Secretary 
for Health Research, Statistics, and Technol- 
ogy, reviewed a number of the activities that are 
ongoing or in the planning stages at the Center. 
These activities are (142): 


1. Comprehensive assessment.—Safety, effi- 
cacy, cost effectiveness, and economic, 
social, and ethical impact of a selected 
technology will comprise a comprehensive 
assessment. Two such assessments will be 
initiated in fiscal year 1980.1 

2. Coverage issues.—HCFA asks the Center 
for advice regarding the appropriateness 
of paying for the use of certain technolo- 
gies with medicare funds. Currently, the 
Center is responding to 53 requests for 
coverage recommendations. 

3. Consensus development processes of NIH. 
—The Center will be more involved in the 
consensus development conferences spon- 
sored by NIH and will cosponsor some of 
these. NCHCT provided an economic 
analysis at the conference on intraocular 
lenses conducted in September 1979. 

4. Intramural activities.—The Center is con- 
ducting cost-effectiveness studies on a 
number of technologies (e.g., intraocular 
lenses, estrogen use by postmenopausal 
women, and antenatal diagnosis). 

5. Overviews.—The Center is writing state- 
of-the-art papers on technologies which 
are candidates for comprehensive assess- 


11n the fall of 1979, NCHCT was handed, or chose, its first tech- 
nology assessment topic—the alpha fetoprotein test kits. FDA is 
delaying approval of this new technology that it has found to be 
safe and effective until the Center completes an assessment of the 
kit's impact on the health care system and society. NCHCT is co- 
ordinating the information, input, and efforts of FDA, the Center 
for Disease Control, NIH, the Health Resources Administration, 
HCFA, and others in order to evaluate the full range of issues in- 
volved. 
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ments (e.g., end-stage renal disease, elec- 
troencephalography, and coronary by- 
pass surgery). 

6. Dissemination.—This activity is just be- 
ginning, with the assistance of NIH and 
FDA among others. In fiscal year 1980, 
the Center will begin its own dissemina- 
tion activities. 

7. Early warning system.—HCFA, the Cen- 
ter for Disease Control, FDA, NIH, and 
the Alcohol, Drug Abuse, and Mental 
Health Administration are developing 
methodologies to identify emerging tech- 
nologies. Non-Federal organizations, such 
as the American Hospital Association, 
will also help identify emerging technolo- 
gies. 

8. Grant and contract program.—This pro- 
gram began in fiscal year 1980 in the area 
of literature syntheses, cost-effectiveness 
studies, and economic, social, and ethical 
analyses. 

9. Centers program.—Public Law 95-623 re- 
quires that three extramural research cen- 
ters be established by September 1, 1981. 


As this list of activities indicates, NCHCT is 
involved in a wide range of technology evalua- 
tion efforts. At one end of the technology eval- 
uation process, the Center is involved in consen- 
sus development activities at NIH which are fo- 
cusing on relatively new medical technologies 
that have areas of uncertainty to be resolved. In 
addition, a very important part of the Center’s 
efforts is focused on the reimbursement system 
(see ch. 5). NCHCT is specifically charged with 
coordinating information and making recom- 
mendations to HCFA regarding new or existing 
medical technologies. All indications are that 
this will be a priority activity of the Center. 
Finally, the Center will provide comprehensive 
examinations of medical technologies that have 
been in use for some time. In many of its func- 
tions, NCHCT has the authority to conduct or 
fund CEAs. 


Potential Use of CEA/CBA 


NCHCT is in a position to act as an informa- 
tion broker to a number of agencies at a variety 
of decision points in the policymaking process. 


Its mandate is so broad, however, that it may 
become overwhelmed by the number and diver- 
sity of functions it is asked to perform. Funding 
levels, and consequently staffing levels, are 
significantly lower than those called for in 
NCHCT’s authorizing legislation.!? This factor 
may significantly affect the number and range 
of duties the Center can be expected to perform. 
So far, HCFA has requested NCHCT to exam- 
ine 53 coverage issues. With this area of respon- 
sibility and the other functions listed above, 
NCHCT will likely find it difficult, at current 
funding and staff levels, to totally fulfill the ex- 
pectations placed on it. 


NCHCT has developed a _ priority-setting 
process that may help it handle the influx of re- 
quests for information, recommendations, as- 
sessments, and general assistance. To a degree, 
priority-setting is vested in the national council, 
but the council focuses primarily on the selec- 
tion of medical technologies that warrant full- 
scale assessments. The remainder of the prior- 
ity-setting process is a mixture of identifying in- 
house needs, perceptions of what areas require 
more immediate action, the assimilation of 
priority areas and views of the agencies NCHCT 
must work with and respond to, and the general 
consensus of the scientific and medical commu- 
nity regarding issues that require attention. In 
addition, NCHCT priorities will be set to a de- 
gree by the deadlines of the agencies with which 
NCHCT works. 


The result of these many factors and consider- 
ations is an unclear picture of how formal CEA/ 
CBA fits into NCHCT’s activities. The agency is 
new; it remains to be seen to what extent 
NCHCT will conduct or support formal CEA/ 
CBA. If, where, and when these techniques are 
employed, it will be interesting to note their 
level of sophistication, their use in decisionmak- 
ing efforts, and their level of input into and im- 
pact upon the policy process. 


2The enabling legislation (Public Law 95-623, sec. 309) author- 
ized $15 million, $25 million, and $35 million for fiscal years 1979 
through 1981, respectively, for NCHCT to carry out its mandate. 
The fiscal year 1979 budget for the Center was $344,000, and the 
fiscal year 1980 budget is $3.25 million. 
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HEALTH SERVICES RESEARCH 


Health services research is relatively new as 
an organized field of research. Only in the last 
two decades has there been significant Federal 
involvement in this field. Before Federal sup- 
port, health services research efforts were spo- 
radic and infrequent. The infusion of Federal in- 
volvement and financial support helped to orga- 
nize the disparate parts into a more focused field 
of inquiry. 


The activities represented by the National 
Center for Health Statistics (see app. B) have a 
history going back at least to 1946 when Con- 
gress first authorized vital statistics activities. 
Two of the Nation’s primary centers for guid- 
ing, conducting, funding, and disseminating 
health services research efforts—NCHSR and 
HCFA—are 12 and 3 years old, respectively.* 
HCFA tends to focus most of its research on its 
program needs. NCHSR, however, was created 
to conduct and support broadly based health 
services research that does not necessarily have 
to be tied to any agency’s program needs. Since 
NCHSR’s arrival on the health services research 
scene, several Federal organizations have been 
authorized to pursue, or have assumed, varying 
degrees of responsibility for health services 
research activities. 14 


The past decade of federally supported health 
services research efforts has raised numerous 
questions regarding the uses and usefulness of 
health services research, its “cost effectiveness,” 
and its impact on the decisionmaking process. A 
great deal of discussion has centered on the fac- 
tors and mechanisms needed to address these 
questions and improve health services research. 
Analysts who have examined many of these is- 
sues have often reached diametrically opposed 


13Although HCFA is 3 years old, the parts that comprise the 
Agency have existed for many years. The medicare and medicaid 
programs and the Office of Research and Statistics were originally 
located in the Social Security Administration. The PSRO program 
at HCFA was preceded by the Health Standards and Quality Bu- 
reau in the Office of the Assistant Secretary of Health at HEW. 

14An Institute of Medicine report identifies at least 15 Federal de- 
partments, agencies, or commissions, that conduct or fund some 
form of health services research. The Federal involvement ranges 
from that of the Federal Trade Commission to that of the National 
Aeronautics and Space Administration and DHHS (formerly 
HEW) (428). 


conclusions regarding the performance of serv- 
ices research. One side of the debate cites nu- 
merous examples of major health policy deci- 
sions where services research was of little conse- 
quence (31,359,383,423). The other side cites 
evidence to support the notion that health serv- 
ices research has been successful in producing 
information useful to the policy process, in 
some instances has been instrumental in stimu- 
lating policy debate and change, and in general, 
compares favorably to other forms of social 
services research (139,186,428,566).1° 


It is doubtful whether the debate surrounding 
the performance and usefulness of health serv- 
ices research will be settled conclusively one 
way or the other. The range and types of issues 
that services research must address, the dynam- 
ic nature of the policy issues and health care sys- 
tem, and the complexity of the decisionmaking 
process combine to almost preclude a conclu- 
sion regarding the relative worth of the informa- 
tion produced by health services research ef- 
forts. Nevertheless, there does seem to be a gen- 
eral feeling that health services research has not 
attained the high level of expectations set for it a 
decade ago and that improvements can be 
made. The ongoing examination and evaluation 
of services research may help improve its focus, 
usefulness, and quality. 


The primary focus of this section is on the 
current or potential use of CEA/CBA in the 
health services research decisionmaking proc- 
ess, i.e., the procedures used to establish health 
services research priorities, fund research proj- 
ects, and evaluate work that is ongoing or that 
has been completed. The section also discusses 
the extent to which certain health services re- 
search agencies support CEA/CBAs as part of 
their research missions. 


NCHSR and HCFA represent the Federal 


Government's most substantial commitment to 
the area of health services research. NCHSR and 


15]t is beyond the intent of this assessment to examine this de- 
bate. For detailed discussions of the history of services research, its 
contributions, its weaknesses, and its development, see the follow- 
ing references listed at the end of this report: 24, 36, 210, 428, and 
643. 
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HCFA provide the highest level of funding of 
health services research in the United States. 
The funding levels for health services research 
have declined in actual and real terms since the 
peak years of the early 1970's. NCHSR’s 1978 
budget represented less than 40 percent of its 
purchasing power for research and training pro- 
grams compared to the levels of the early 1970's 
(428). Together, NCHSR and HCFA contribute 
roughly 40 percent of the total amount of Feder- 
al funds allocated to health services research 
(428). In fiscal year 1980, they will control 
approximately $50 million in moneys ear- 
marked for services research. These agencies oc- 
cupy a very influential position in the health 
services research community and are in a posi- 
tion to exercise considerable influence on the 
content, direction, and level of health services 
research in this country. 


It is unclear where applied biomedical re- 
search ends and where health services research 
begins. Several agencies that focus primarily on 
conducting and supporting basic and appied re- 
search routinely conduct or fund health services 
research as part of their programmatic missions 
(428). 


Health services research is unlike most other 
areas of scientific inquiry in that it is not orga- 
nized around a single discipline with unique per- 
spectives, closely drawn areas of expertise, com- 
mon methodologies or techniques, and standard 
nomenclatures. Health services research is a 
mixture of concepts, methodologies, attitudes, 
and professions that could easily span a large 
university's graduate school catalog. The field 
of health services research must accommodate 
data, methodological frameworks, disciplines, 
and perspectives from the diverse fields of medi- 
cine, other health-care-related disciplines (epi- 
demiology, nursing, public health, etc.), bio- 
statisticians, engineers, lawyers, demographers, 
geographers, operation researchers, economists, 
social workers, hospital and business adminis- 
trators, and so forth. Individual health services 
researchers tend to approach the issues from the 
confines and perspective of their particular 
discipline. 


David Mechanic describes the purpose of 
health services research as follows (396): 


The health services research field focuses on 
the production, organization, distribution, and 
impact of services on health status, illness, and 
disability . . . it concentrates attention on im- 
proving the distribution, quality, effectiveness, 
and efficiency of medical care. 


A study by the Institute of Medicine on health 
services research attempted to provide guide- 
lines for the description or classification of the 
types of health services research that exist. Ac- 
cording to that report, a study is classified as 
health services research if it satisfies two criteria 
(428): 


1. It deals with some features of the struc- 
ture, processes, or effects of personal 
health services. 

2. At least one of the features is related to a 
conceptual framework other than that of 
contemporary applied biomedical science. 


National Center for 
Health Services Research 


NCHSR was created in 1968 without explicit 
congressional authorization. It was not until 
1974 that NCHSR received legislative authority 
via the Health Services Research, Health Statis- 
tics, and Medical Libraries Act (Public Law 
93-353). Since then, several laws have added to 
or modified NCHSR’s research domain. ?® 


NCHSR has two principal responsibilities. 
One is to develop information that might be 
used by various decisionmakers in the public 
and private sectors. The other is to ensure that 
the information that results from the research, 
evaluation, and demonstration activities of 
NCHSR is disseminated rapidly and in a form 
that is usable. 


NCHSR is a major supporter of broadly fo- 
cused health services research. Its fiscal year 
1980 operating budget will be almost $30 mil- 
lion. NCHSR is almost unique in the Federal 
Government in that it can sponsor health serv- 
ices research apart from direct administrative or 


1°A few of those laws are the National Health Planning and Re- 
sources Development Act of 1974 (Public Law 93-641), the Emer- 
gency Medical Services Systems Act of 1973 (Public Law 93-154, 
sec, 1205), and the Health Maintenance Organizations Act of 
1973: These and others have influenced the direction of research 
priorities and level of funding from NCHSR. 
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programmatic needs. It is not responsible for the 
administration of any health care delivery or re- 
imbursement activities; it exists solely to con- 
duct and sponsor health services research and to 
disseminate the results of that research to rele- 
vant Government agencies, the research com- 
munity, and other interested parties. 


The NCHSR statute’s language is so broad 
that the Agency retains considerable leeway in 
its selection of specific research issues to pursue. 
To identify the areas of research it needs to con- 
duct and support, NCHSR employs an informal 
consensus development process. 


Priority Setting 


The priority-setting process involves several 
steps. First, NCHSR canvasses policymakers, 
consumers, Government and non-Government 
experts, health care providers, professional as- 
sociations, program administrators, and others, 
in an attempt to identify current and emerging 
issues that present the most immediate problems 
to the health care system. In the first cut at the 
list of suggested issues, a number of criteria are 
used, two of which are the relative importance 
of the issue and the perception that there is a 
good chance that the research will provide in- 
formation that will contribute substantively to 
the policy process. 


NCHSR staff, as well as outside professionals 
involved in various research areas, are involved 
in the culling process. The Director of NCHSR 
selects the top priority concerns from the major 
issues identified by this process. Before the list is 
adopted, it is submitted to an assorted group of 
NCHSR members and non-NCHSR experts for 
review. The priority issues that emerge from 
this process become the areas of health services 
research that NCHSR focuses on. The issue se- 
lection process is not a one-time occurrence, but 
rather an ongoing interaction between NCHSR 
and the health care community. 


The list of priority areas does not lock out all 
other health care issues. NCHSR can, and does, 
consider “meritorious and potentially impor- 
tant” proposals for a range of nonpriority is- 
sues. In-house, or intramural, research can also 
pursue areas of interest that fall outside the pri- 
ority areas. 


Currently, NCHSR supports five general pri- 
ority areas and a special studies category that is 
concerned with examining issues of emerging 
importance or of research methodology. The 
current priority areas are (438): 1) health care 
costs and cost containment, 2) health insurance, 
3) planning and regulation, 4) technology and 
computer science applications, and 5) health 
manpower 


Evaluation of Research Proposals 


NCHSR’s health services research efforts are 
allocated among intramural research, extra- 
mural research grants and contracts, and center 
grants. Intramural research is subject to an in- 
formal review process within NCHSR. When 
the need arises, staff proposals for in-house 
research projects may also be reviewed by non- 
Federal experts. Projects are formulated accord- 
ing to special research needs or personnel capa- 
bilities at NCHSR. The in-house project pro- 
posal moves through the administrative hier- 
archy, receives comments and suggestions from 
the staff, and may be circulated to outside re- 
viewers before final action is taken by NCHSR. 
Health services research centers receive funding 
apart from the extramural and intramural re- 
search budget. NCHSR’s authorizing legislation 
called for the funding of at least six “existing and 
new centers for multidisciplinary health services 
research, evaluation, and demonstrations.” 
There are eight such centers currently in opera- 
tion (650). 


NCHSR uses a dual review type of evaluation 
process to screen and select grant and contract 
applications it receives (385). This process is 
quite similiar to the peer review system de- 
scribed earlier for NIH. Grant and contract pro- 
posals that exceed $35,000 in direct costs are re- 
quired to be reviewed for scientific and technical 
merit by study sections composed of non-Feder- 
al Government experts. Proposals that do not 
exceed $35,000 are evaluated by NCHSR staff, 
and where needed outside reviewers. 


Explicit CEA/CBAs are not used in the prior- 
ity-setting, project selection, or research evalua- 
tion processes. A few of the project criteria used 
in the selection process, however, make it likely 
that there will be significant emphasis on the 
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relationship between the application’s budget 
(cost) and its potential outcomes (benefits). 
NCHSR receives far more applications than it 
has the funds to support, so in the project selec- 
tion process, it does consider cost and effec- 
tiveness. The budget and outcomes criteria used 
to evaluate applications are considered with ref- 
erence to the Agency’s budget and goals. 


CEA/CBA as Part of NCHSR’s 
Research Mission 


NCHSR supports a number of cost-effective- 
ness and cost-benefit studies that cover a broad 
range of health services issues (439) and is the 
major supporter of CEAs and CBAs in the Fed- 
eral health care research system at this time. 
Two of NCHSR’s five research priority areas— 
health care costs and cost containment, and 
planning and regulation—specifically call for re- 
search using CEAs to examine the issues within 
these priority areas (438). 


The CEAs and CBAs that NCHSR supports 
range from rigorous analyses of specific health 
care topics to broad studies of more complex 
health care issues. The results of these studies, 
like most of NCHSR’s research products, are 
circulated via the Agency’s formal publications 
dissemination process, conferences, seminars, 
journals, announcements, etc. NCHSR is in a 
position to monitor the eventual use of the re- 
search results by other agencies and decision- 
makers. In most cases, though, the nature of the 
policymaking process makes it quite difficult to 
determine to what extent a given piece of in- 
formation is used to reach a final policy deci- 
sion. NCHSR is powerless to assure that the re- 
search results will actually be used in the deci- 
sionmaking process. The Agency’s legislative 
mandate is to support health services research 
projects that answer, or at least address, the 
issues that the research community and policy- 
makers feel are important. Decisions about 
whether and, if so, how to use the information 
that is generated are left up to agencies and deci- 
sionmakers outside NCHSR. 


Health Care Financing Administration 


HCFA, established in 1977 as a result of a ma- 
jor reorganization at the Department of Health, 


Education, and Welfare (now DHHS), is a re- 
cent addition to the Federal Government's 
health services research community. This Agen- 
cy is the organizational center that administers 
medicare/medicaid programs, the Professional 
Standards Review Organization (PSRO) pro- 
gram, and the research and statistics programs 
for these areas. 


The legislation authorizing these programs al- 
lows HCFA to consider and pursue a range of 
health services research topics.1? HCFA is par- 
ticularly responsible for sponsoring research 
that relates to its primary mission: the adminis- 
tration and evaluation of the medicare/medic- 
aid area and the PSRO function. The mandate 
of the Office of Policy Planning and Research at 
HCFA, however, is broad enough to include a 
wider range of research topics.*® 


HCFA currently has five major priority areas 
of grant support. Specifically, Agency support 
is focused on projects that meet one of the fol- 
lowing criteria (284): 


1. develops or demonstrates new financing 
mechanisms for health care services; 

2. utilizes financing mechanisms to influence 
the effectiveness or delivery of health care 
services; 

3. develops or demonstrates management or 
administrative procedures that will benefit 
HCFA programs; 

4. develops knowledge or undertakes anal- 
yses of the basic nature and structure of 
health care costs and factors that affect 
their rate of increase; or 

5. examines the economic and _ behavioral 
relationships between the financing of 
health care services and the total activities 
of the health care sector of the economy. 


The legislation that establishes HCFA’s scope of responsibility 
is the Social Security Amendments of 1972 (Public Law 92-603). 
Sections 222 and 245 of this law provide for health services re- 
search and demonstration projects for a variety of reimbursement 
issues, performance incentives, etc. The National Health Planning 
and Resources Development Act of 1974 (Public Law 93-641) 
added provisions for work in ratesetting and development of qual- 
ity data for providers. 

18The Office of Policy Planning and Research of HCFA grew out 
of the Office of Research and Statistics (ORS), formerly of the So- 
cial Security Administration. ORS became a major source of sup- 
port for health services research in the late 1930's. 
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Priority Setting 


HCFA’s priority research areas are chosen in 
an informal process much like the one used by 
NCHSR. Opinions, suggestions, and recom- 
mendations are solicited from a variety of peo- 
ple with a broad background of experience and 
training. Their recommendations are culled to 
form a list of research priorities. Although pri- 
ority areas receive special attention in the selec- 
tion process, new ideas and innovations are not 
automatically excluded from consideration if 
they fall outside these areas. Relevant examples 
of the issues on which HCFA is focusing are hos- 
pital costs, physician reimbursement, and the 
quality and effectiveness of various health care 
areas. 


Evaluation of Research Proposals 


The peer review mechanism and the project 
evaluation process at HCFA are very similiar to 
the systems at NCHSR. The major difference be- 
tween the two approaches is the formal involve- 
ment of HCFA staff in the initial technical re- 
view panels. Instead of being composed of all 
non-Federal experts like NCHSR’s and NIH’s 
panels, HCFA’s review panels are composed of 
an equal mixture of HCFA staff, non-Federal 
experts, and non-HCFA Government experts 
(284). The criteria used to evaluate the merit of 
the research application are geared as much to 
HCFA’s program needs as to technical and sci- 
entific merit. Final review and funding decisions 
are made by HCFA staff and administrators. 


Use of CEA/CBA 


HCFA is like the other agencies discussed 
above, in that it does not use explicit CEA/CBA 
to select research goals or grant proposals, or to 
evaluate project outcomes. Awareness of proj- 
ects’ “cost effectiveness” to the Agency's goals, 
however, is present. 


HCFA focuses much of its substantive re- 
search on priority issues that try to determine 
the relative efficiency of various methods of 
delivering care. Many of its research solicitation 
areas are for work to arrive at CEA/CBA-like 


evaluations of specific issues. Or, data are pur- 
sued that might assist HCFA in determining cost 
and effectiveness measures. Several of the proj- 
ects recently completed, as well as a number of 
ongoing efforts, are directly focused on cost- 
effectiveness issues (283). 


The CEA/CBAs that HCFA is supporting are 
similar to those at NCHSR, in that they range 
from fairly rigorous attempts at examining the 
costs and effectiveness of a medical technology 
to being closer to effectiveness studies that in- 
clude costs as an analytical afterthought, if at 
all. The information produced as a result of 
these analyses is combined with the other infor- 
mation and considerations that are part of 
HCFA’s program responsibilities. At HCFA, as 
at the agencies previously described, the sup- 
port, evaluation, and use of analyses or infor- 
mation in general is not usually a linear process. 
It is extremely difficult to point to a given piece 
of information, a cost-effectiveness study for in- 
stance, and determine where, how, or even if, 
the knowledge gained from a given research 
project was directly incorporated into a given 
policy decision. Nevertheless, since HCFA can 
tailor its use of CEA/CBA to suit its special pro- 
gram needs and might then be able to implement 
that information within its organizational 
boundaries, it could possibly serve as an in- 
structive example of the support and use of 
CEA/CBA in the decisionmaking process. 
HCFA is in a position to evaluate the impact of 
its actions after programs or program changes 
have been in place for a period of time. NCHSR 
and NCHCT do not share this type of start-to- 
finish authority. NCHSR and NCHCT have 
more limited access and input to the decision- 
making process than do HCFA, NIH, and other 
health care agencies that maintain program re- 
sponsibilities. 


Although HCFA is a collection of fairly ma- 
ture programs that have been reorganized into a 
new agency, the organization has not fully set- 
tled into its new niche. It will be interesting to 
note how, or if, HCFA uses any CEA/CBAs 
that are produced by HCFA’s own research sup- 
port system, by NCHSR, or via the new efforts 
of NCHCT. 
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USE OF CEA/CBA IN R&D PROGRAMS: GENERAL FINDINGS 


Formal CEA/CBAs are not used or supported 
to a great extent in the health care R&D system. 
At the process (or administrative) level, Federal 
agencies rarely use explicit CEA/CBA as a deci- 
sion-assisting tool. In only a few cases has 
CEA/CBA been instrumental in facilitating a 
given allocational or policy decision. (See ‘Sur- 
vey of Agency Use of CEA/CBA,” app. B.) The 
agencies do seem to make efforts to employ a 
“cost-effective” approach and attitude towards 
the decisions that are made regarding the setting 
of priorities, selection of research projects, and 
in the evaluation of research products. These at- 
titudes and approaches are transferred into ac- 
tion primarily via external and internal peer 
review mechanisms. At the substantive level, 
several formal CEAs and CBAs have been or 
will be supported by NIH, NCHCT, NCHSR, 
and HCFA. The research done at this level may 
feed back to a degree into the administrative de- 
cisionmaking and evaluation processes at these 
agencies. A significant amount of the CEA/ 
CBA research will be used to add to the body of 
knowledge in the health care system. 


Efficiency-based techniques such as CEA/ 
CBA have not gained a substantial foothold in 
the R&D decisionmaking process. The same can 
also be said with respect to the other formal 
methodologies that have received some use and 
recognition in the field of policy analysis (i.e., 
decision analysis, operations research, technol- 
ogy assessment, risk-benefit analysis, etc). The 
relatively infrequent reliance on these types of 
analyses stems from a number of factors related 
to the techniques and the nature of the health 
care system in general and the R&D process in 
specific. The complexity and uncertainty of the 
R&D endeavor and its eventual products does 
not lend itself well to the constraints of efficien- 
cy-based methodologies. This may be the most 
important reason for the lack of use of CEA/ 
CBA in R&D. The health care R&D process is 
extremely complex, essentially political, and 
quite often is passive. The Federal R&D process 
is authorized, funded, and supervised by repre- 
sentatives of the public and is under the scrutiny 
of the Nation as a whole. The R&D system must 
be responsible and responsive to the changing 


needs and goals of the country; hence, it is a 
highly political process. Finally, the system 
tends to be passive. Many problems are often 
presented to the decisionmaker with the objec- 
tives preordained and the viable options avail- 
able to attain the goals few in number. The R&D 
process is fueled by the imagination and initia- 
tive of those researchers outside the decision- 
making process. A research goal or national 
health policy objective can be established, but 
the system does not move without the initiative 
of those who must create and submit research 
ideas and plans to the funding agencies. To a 
significant degree, R&D-supporting agencies 
must wait for, and react to, the ideas, sugges- 
tions, efforts, and research findings of the 
thousands of health care researchers both within 
and outside Federal Government. It is extremely 
difficult for CEA/CBA to predict, evaluate, or 
include the importance of the dynamic aspects 
of R&D. As a result, CEA/CBA’s credibility, 
usefulness, and input to the R&D decisionmak- 
ing process are limited. 


CEA/CBA is most supported and used at the 
health services research end of the R&D spec- 
trum and least supported and used at the basic 
and applied end. This situation follows logically 
from the inability of formal analytical tech- 
niques to adequately deal with the high level of 
uncertainty that is part of the technology R&D 
process. Predictions, opinions, and “guessti- 
mates” are the tools of the trade in this area. 
One can include considerations of uncertain fac- 
tors in CEA/CBA,; sensitivity analysis can help 
to an extent. But the level of uncertainty re- 
mains high. 


The methodological shortcomings of CEA/ 
CBA techniques are compounded by the atti- 
tudes of many of the potential users of CEA and 
CBA in the R&D system. The decisionmakers’ 
perceptions regarding the usefulness and validi- 
ty of CEA/CBA are such that many doubt that 
these techniques are either necessary or helpful 
in much of the R&D context. This view is par- 
ticularly strong at the basic and applied research 
level. This situation should not be surprising. 
Until recently, cost containment and cost-effec- 
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tiveness criteria were not heavily stressed in 
health care research. NIH has traditionally fo- 
cused on the quality of the research supported 
and the safety and efficacy of the technologies 
developed as results of research efforts. Finding 
cures for health care problems, not saving 
money, has been the primary goal. At the health 
services end of the R&D spectrum, distribution, 
cost, and quality of care have received needed 
attention. Cost and effectiveness criteria have 
been part of the R&D efforts; but only parts of a 
much larger focus. Recently, decisionmakers 
have been asked to make them a larger part of 
their decisionmaking criteria. As a result, in- 
creasing numbers of decisionmakers are becom- 


ing aware of the uses (and possible abuses) of 
CEA and CBA techniques. 


Another impediment to the use and support 
of CEA/CBA is the presence of other decision- 
assisting techniques that have been fairly suc- 
cessful in guiding and informing the decision- 
making process. Peer review panels, publication 
of results in reputable journals, advisory coun- 
cils, conferences, and other mechanisms have all 
worked fairly well as “quality” controls and, to 
an extent, as cost-effectiveness filters. These 
processes are firmly in place, have performed 
reasonably weli over the years, and have by and 
large produced commendable results. The pres- 
sure to maintain these existing support systems 
might tend to overwhelm any serious effort to 
incorporate CEA/CBA in the mainstream of the 
decisionmaking process. 


The organizational and statutory frameworks 
are currently in place to allow the use of CEA or 
CBA in the decisionmaking efforts of the R&D- 
supporting agencies. The limiting factors are the 
perceived need for and usefulness of the infor- 
mation that might result from CEA/CBA re- 
search. 


If CEAs could be adapted to the need of bio- 
medical R&D, the use of CEA-type studies at 
NIH might logically occur at the late transfer 
stages of a technology's movement from the lab 
to the clinic and at the consensus development 


meetings held by NIH and NCHCT. NIH may 


also be able to incorporate some form of effi- 
ciency-based analysis in its center, contract, and 
intramural research efforts. NIH has more con- 
trol over the formulation, direction, and eval- 
uation of these types of research efforts than it 
does over extramural project grant research ef- 
forts. It is at these points in the decisionmaking 
and information-gathering processes that the 
CEA/CBA technique might help serve the needs 
of the NIH decisionmakers. NIH has in a very 
few instances used CEA techniques to evaluate 
the ongoing or completed programs and proj- 
ects that are performed as adjuncts to the basic 
and applied research missions (e.g., health edu- 
cation programs, disease prevention advertising 
programs, information dissemination projects, 
screening programs, etc.). It might consider the 
usefulness of increasing its efforts along these 
lines. Once again: Doing CEA/CBA for any 
reason requires that the limits and usefulness of 
the analyses be kept in mind. 


NCHCT could provide input into a range of 
decision points in the R&D process. Its legisla- 
tive mandate authorizes the use of CEA/CBA to 
examine newly emerging and existing technol- 
ogies. NCHCT cosponsors consensus develop- 
ment conferences at NIH and is charged with 
providing information to HCFA on reimburse- 
ment issues concerning medical technologies. It 
remains to be seen how NCHCT will use 
CEA/CBA in the decisionmaking process. 


NCHSR and HCFA hold promise as support- 
ers of CEA/CBA research. HCFA might also be 
a user of CEA/CBA information in its PSRO 
and medicare programs. These two agencies 
focus on technologies in later R&D stages and 
on the services end of the health care system. 
CEA/CBA has recently received increased at- 
tention in this area of research and one sees in- 
creasing use of this technique to examine a host 
of services research issues.’° 


19Such use is discussed at greater length in Background Paper #1: 
Methodological Issues and Literature Review, prepared by OTA in 
conjunction with this assessment. 
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10. 


Health Maintenance Organizations 


INTRODUCTION AND BACKGROUND 


Health maintenance organizations (HMOs) 
have been described by their advocates as a 
“cost-effective” way to provide health care 
(489). It appears that HMOs do provide care to 
their enrollees at varied but substantial cost sav- 
ings: Empirical evidence shows that HMO en- 
rollees pay in the range of 10 to 40 percent less 
in total costs (premium plus out-of-pocket costs) 


than conventionally insured comparison groups 
(370).* 


Explanations of HMO performance, whether 
measured by costs, access, quality, or physician 
or enrollee satisfaction, however, are a hybrid 
of rhetoric, theory, and evidence. There is very 
little information available on the details of how 
HMOs actually function, especially on the ana- 
lytical tools they use to make internal decisions 
on how to allocate their resources (295). This 
chapter presents some preliminary evidence re- 
garding the current and potential use and useful- 
ness of cost-effective analysis/cost-benefit anal- 
ysis (CEA/CBA) in resource allocation decision- 
making in HMOs. It does not, however, exam- 
ine directly the question of the cost effectiveness 
of HMOs themselves. 


HMOs, representing the prepaid segment of 
the health care market, are both insurors and 
providers of health care (300). Their revenue 
comes from capitation payments, and they are 
responsible for delivering care to an enrolled 
population (371). Theoretically, the HMO has a 
direct economic incentive to provide “cost- 
effective” care. Because of this, some people 
have assumed that the HMO might be more re- 
ceptive than conventional health care delivery 
settings to the use of CEA/CBA (615). 


It is important to note, however, that HMOs 
exist in a predominantly fee-for-service environ- 


1This range reflects the differences observed between types of 
HMOs. Prepaid group practices consistently show the lowest total 
(premium plus out-of-pocket costs), though there is much varia- 
tion among them; individual practice associations show among the 
highest costs (372). 


ment. Since potential enrollees have a choice of 
providers, and physicians a choice of practice 
settings, the HMO must compete for both. Be- 
cause its financial viability depends on its ability 
to attract and retain enrollees (as well as physi- 
cians), the HMO will be induced to offer bene- 
fits and services comparable to those offered by 
its fee-for-service competitors. 

While the distinctive characteristics of the 
HMO guarantee control of total costs (by virtue 
of capitation payments) and promote efficiency 
(by virtue of the HMO’s responsibility to deliver 
appropriate services to an enrolled population 
within a constrained budget), there is nothing to 
require the HMO to be any more or less con- 
cerned with the effectiveness and benefits of a 
service than are providers in other health care 
delivery settings: Rather, the benefits side of the 
CEA/CBA problen: is largely accepted as being 
defined by the norms of “good medical practice” 
that prevail in the marketplace. The expecta- 
tions of HMO enrollees and physicians tend to 
diminish the importance of resource allocation 
questions pertaining to whether or not a benefit 
or service should be offered. 

In general, the HMO plan bears primary 
responsibility for the financial viability of 
the HMO (in a management or administrative 
sense). The plan faces explicit resource alloca- 
tion decisions upon which economic incentives 
and financial constraints directly bear. Whether 
or not physicians are sensitive or responsive to 
these incentives and constraints in making their 
implicit resource allocation decisions, however, 
is the subject of considerable debate (152,371, 
589,590). Some claim that physicians are un- 
aware of incentives and constraints, and even 
point to complete coverage as giving them a free 
hand (372). Yet others assert that physicians in 
the HMO setting are sensitive to costs (at least 
on a subliminal level) and that even a decision, 
for example, to treat bronchitis, or to perform a 
coronary bypass, specifically involves the 
aspect of costs (619). 
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In fact, the available evidence shows that 
almost all of the observed cost savings in HMOs 
are attributable to lower rates (by 30 percent) of 
hospitalization for HMO enrollees (373). Deci- 
sions concerning the need for hospitalization of 
HMO enrollees are primarily made at the discre- 
tion of attending physicians. It is true that the 
resource allocation decisions made at the ad- 
ministrative level will define the parameters of 
decisions faced by HMO physicians. Apart 
from direct and indirect controls imposed by 
the plan of the HMO, however, there is little 
consensus on any one explanation of why HMO 
physicians should exhibit this apparent “cost 
consciousness.” HMO physicians, trained no 
differently than physicians practicing in the fee- 
for-service environment, bring with them habits 
and values acquired in medical school or in 
some other prior practice setting (300). Theo- 
retically, there is no reason to expect that HMO 
physicians would be any more likely than their 
fee-for-service counterparts to explicitly consid- 
er the aspect of cost, or beyond that, to consider 
cost concomitantly with the effectiveness of a 
service. 


If anything, the HMO physician can be ex- 
pected to allocate the available resources to a 
given medical problem foremostly on the basis 
of effectiveness and/or efficacy, differing in this 
respect from the plan, which will be motivated 
to allocate resources in significant part on the 
basis of costs, i.e., efficiency. No further at- 
tempt will be made in this chapter to investigate 


the use and/or usefulness of CEA/CBA in the 
decisionmaking of HMO physicians. 


The preliminary evidence and conclusions 
that are presented in the discussion that follows 
pertain largely to the resource allocation deci- 
sions made at the organizational, i.e., adminis- 
trative, level of the HMO plan. The evidence 
discussed in this chapter should be regarded as 
tentative because of the great diversity in types 
of HMOs and the small number of HMOs from 
which the evidence was gleaned. The influences 
assumed to determine the resource allocation 
questions relevant to the HMO theoretically 
derive from “generic” characteristics of HMOs 
that distinguish them from other health care 
providers. It is important to note, however, that 
there is probably no such thing as a “typical” 
HMO. There is a tendency (in the rhetoric and 
limited literature available on HMOs) to jux- 
tapose HMOs against “other providers,” but 
this has masked important differences among 
HMOs (295). 


The findings presented here do seem to sub- 
stantiate the preliminary conclusion that formal 
CEA/CBA, except in its “net cost analysis” 
forms, is not used to any significant degree in 
decisionmaking in HMOs. In large part, the evi- 
dence is based on: 1) actual resource allocation 
decisions recently made in a handful of HMOs 
around the country, 2) the analytic techniques 
those HMOs used, and 3) the role that analysis 
played in those particular decisions. 


LEGISLATIVE HISTORY AND FEDERAL SUPPORT 


HMOs have occupied a prominent position in 
Federal health policies during the last decade. 
They have been promoted as one strategy for 
controlling health care costs and encouraging a 
more rational allocation of resources to health 
care needs (295).2 Much of HMOs’ original and 


2In 1979, the Office of Health Maintenance Organizations 
(OHMO) contracted for a CBA to examine the economic costs and 
benefits of continued Federal assistance to new HMO develop- 
ment. That study determined the estimated rate of return (in cost- 
savings to the community) derived from the Federal HMO devel- 
opment investments. It found that Federal assistance costs are re- 


sustaining appeal derives from the fact that 
these organizations have been viewed as a more 
desirable alternative than Federal regulation for 
achieving those objectives (95). 


The term “health maintenance organization” 


was the brainchild of the Federal HMO initia- 


covered in the form of community health care cost savings after 8 
years of HMO operation, and projected even more substantial fu- 
ture savings (302). Based on these findings, OHMO has devised a 
10-year strategy for focusing Federal funds and support of HMO 
sre in those areas that offer the greatest potential return 
489). 


tive in the early 1970’s,? but the concept of pre- 
paid group practice was born under private 
sponsorship and developed without Federal as- 
sistance before 1970 (372, 489). In 1973, when 
the original Health Maintenance Organizations 
Act was legislated (Public Law 93-222), there 
were 39 prepaid group practices in existence, 
some of which had been providing care since the 
late 1920's (489). A few large programs, such as 
Kaiser Foundation Medical Care Plan (then, as 
now, the Nation’s largest HMO), accounted for 
the bulk of enrollment. 


Based on the performance of these “proto- 
type” HMOs already in operation, advocacy of 
the HMO approach to health care became a ma- 
jor Federal initiative in 1971 (589). Federal ef- 
forts culminated in the enabling legislation for 
what was to be a 5-year demonstration pro- 
gram. This legislation (Public Law 93-222) pro- 
vided a congressional mandate and Federal re- 
sources to demonstrate the’. . . extent to which 
the HMO concept can be transplanted to new 
environments with new participants putting it 
into operation.” The authority has been ex- 
tended twice, once in 1976 (Public Law 94-460) 
and again in 1978 (Public Law 95-559) (489). 


Federal legislation provides explicit directives 
on how an HMO should be organized and oper- 
ated. According to Federal law, HMOs are re- 
quired to provide both ambulatory and hospital 
services. The law recognizes three variations in 
organization that distinguish three major types 
of HMOs (discussed below). A stringent and 
precise definition of the federally qualified 
HMO mandates a very comprehensive benefits 
package (including such benefits as mental 
health, dental, alcohol treatment, and family 
planning services), sets limits on charges related 
to utilization (in addition to premiums), and 
specifies enrollment mechanisms and the extent 


3Paul Ellwood is credited with coining the term “health mainte- 
nance organizations” in 1973 to encompass the prepaid group 
practices, foundations for medical care (otherwise know as indi- 
vidual practice associations), and other types of prepaid systems of 
health care delivery (174). 
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of risk sharing (369,640). Of the 212 HMOs in 
operation as of April 1979, only 99 (47 percent) 
qualified under this strict Federal definition 
(489). 


Critics of the legislation have viewed it as un- 
fairly restrictive, citing a lack of comparable re- 
strictions in the health insurance industry (372). 
Still, increasing levels of funding have been au- 
thorized during the past decade for the con- 
tinued Federal support of new and developing 
HMOs. Approximately 97 (45 percent) of the 
212 HMOs operational in April 1979 had re- 
ceived some sort of Federal financial assistance. 
Of the 99 federally qualified HMOs, 74 (75 per- 
cent) have received Federal assistance. The 
Federal HMO program is responsible for a great 
deal of growth in prepaid plans over the past 
decade (489). In June 1979, there were 215 
HMOs in the United States with a total enroll- 
ment of 8,226,000 persons. The consistent pro- 
motion of the HMO program by Congress re- 
flects the opinion that Federal support of HMOs 
can yield substantial national benefits. 


In September 1979, Congress passed amend- 
ments (Public Law 96-620) to the Health Plan- 
ning Act (Public Law 93-641) that effectively ex- 
empt HMOs from the certificate-of-need (CON) 
regulations administered by local health systems 
agencies (HSAs). The exemption, based on evi- 
dence that CON regulations were being used to 
delay or prevent HMO development (372), re- 
flects congressional support of the broader ra- 
tionale that HMOs may generate competition 
among health care providers. As boldly stated 
by the Office of Health Maintenance Organiza- 
tions (OQHMO) in the Department of Health and 
Human Services (DHHS) (489): 


. . » HMOs have proven to be a quality, cost- 
effective, competitive alternative to traditional 
fee-for-service forms of medical practice... 
Both Federal support and substantial private 
initiative have coalesced over the past several 
years to generate a new momentum for future 
HMO growth... It appears that HMOs are 
moving into the mainstream of the health deliv- 
ery system of this country. 


126 © The Implications of Cost-Effectiveness Analysis of Medical Technology 





DEFINITIONS 


As indicated in the introduction, there is 
probably no “typical” HMO. That notwith- 
standing, however, all HMOs do share some 
characteristics that distinguish them from other 
health care delivery systems. 


A “generic” definition identifies a minimum 
set of common denominators that are distinctive 
characteristics of HMOs (372).4 One such def- 
inition identifies the following as HMOs’ com- 
mon characteristic(s): “the provision of compre- 
hensive services by a defined set of physicians to 
a voluntarily enrolled population paying a pro- 
spective per capita fee” (95). Luft has recently 
developed and elaborated on a five-point defini- 
tion of the “generic” HMO (372). The five crite- 
ria he identifies are: 


1. The HMO receives a fixed annual (or 
monthly) premium from enrollees that is 
independent of their actual use of services. 

2. The HMO assumes at least part of the fi- 
nancial risk (or gain). 

3. The HMO serves a defined population. 

4. The HMO assumes a contractual responsi- 
bility to provide services to its enrollees. 

5. HMOenrollees are voluntarily enrolled. 


Since each of these criteria is subject to being 
met to a greater or lesser degree, Luft’s defini- 
tion allows for substantial variation among 
HMOs that meet these requirements of the ‘ge- 
neric’” HMO. The extent to which these criteria 
are met can be used to distinguish between 


HMOs. 


Since the “generic” definition encompasses 
the two or three major types of HMOs generally 


‘This discussion draws heavily from a draft of the forthcoming 
book by Harold Luft entitled Dimensions of HMO Performance 
(372). OTA wishes to especially acknowledge Dr. Luft for this 
helpful conceptual definition of HMOs, of which there are (in real- 
ity) virtually limitless varieties. 


recognized>—the prepaid group practice (PGP), 
including the group and staff models, and the 
individual practice association (IPA) model— 
additional criteria are required to distinguish be- 
tween these types.° 


Two additional criteria that allow the distinc- 
tion between PGP and IPA types to be made 
are: 1) how the physicians are paid by the 
HMO; and 2) whether they provide services to 
enrollees in the context of an organized group 
(closed panel), or from a number of independent 
offices (open panel). In the PGP model, physi- 
cians are paid on either a salary or capitation 
basis. In contrast, IPA physicians are paid by 
the HMO on a fee-for-service basis (according 
to a fee schedule) for service received by HMO 
enrollees. Enrollees in the IPA model HMO 
have their choice of physicians from those be- 
longing to the IPA. In practice, the PGP type of 
HMO clearly predominates: In June 1979, about 
63 percent of all HMOs were PGPs, but PGPs 
served about 84 percent (or 6,942,000) of the 
over 8 million total HMO enrollees (300). The 
distinction between group and staff model PGPs 
is based on differences in the nature of the rela- 
tionship between the physicians group and the 
plan. In the group model PGP, the physicians 
are a distinct legal entity that contracts with the 
plan of the HMO, whereas in the staff model 
PGP, the physicians are hired “on the staff” of 
the HMO. 


°A fourth kind of HMO, the network model, is also recognized 


by some, but is not legally recognized (in the HMO statute) as are 
the prepaid group and staff models, and the IPA models (640). The 
Health Insurance Plan of Greater New York is the oldest, largest, 
and best known of the network models. Though network models 
are currently fewest in number, the model is gaining popularity, 
primarily because of its attractiveness to physicians and consumers 
alike, while maintaining a competitive posture in the community 
(413). 

*These characteristics are: 1) the method of paying the key deci- 
sionmakers, 2) whether physicians are full or part time, 3) the ex- 
tent of risk sharing, 4) whether the HMO is non- or for-profit, 5) 
whether the HMO controls its hospital, 6) whether there are econ- 
omies of scale, and 7) the competitive market environment faced 


by the HMO (372). 
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INCENTIVES, CONSTRAINTS, AND IMPERATIVES 


There is considerable speculation on the rela- 
tive influence on resource allocation questions 
relevant to the HMO of the internal incentives 
and budget constraints and of the external mar- 
ket environment. These two influences derive 
from the five distinctive characteristics of the 
“generic” and major types of HMO. The inter- 
nal economic incentives and constraints arise 
from the combined features of prepayment, fi- 
nancial risk, and defined population. The exter- 
nal competitive imperative stems from the fea- 
ture of voluntary enrollment. The contractual 
responsibility of the HMO to provide services 
has a dual effect, acting in concert with both sets 
of characteristics to reinforce their respective in- 


fluences on resource allocation questions rele- 
vant to the HMO. 


Economic Incentives and Financial 
Constraints 


The key feature of prepayment implies two 
things for the HMO: 1) that it has a fixed or con- 
strained budget (total revenue being determined 
by the number of enrollees and the fixed premi- 
um rate), and 2) that the premium paid by 
HMO enrollees is independent of their actual 
use of services. Thus, the usual financial incen- 
tives that exist in fee-for-service practice, in 
which the net income of the provider is directly 
related to the services rendered, are reversed. 
Within the constraint of a fixed payment per 
member per month, an HMO'’s net income, to 
some extent, is related to the number of services 
provided. Consequently, there is a direct finan- 
cial incentive to provide services more efficient- 
ly, or to provide fewer services (presumably re- 
ducing unnecessary or inappropriate services), 
since this will translate into lower premiums to 
enrollees or higher profits to the HMO, or both 
(615). 


The economic incentives and financial con- 
straints deriving from prepayment are enforced 
by the second criterion which says that the 
HMO assumes at least part of the financial risk 


(or gain) in the provision of services. As far as 
the “generic” HMO is concerned, this feature 
only implies that some portion of the burden of 
risk must be borne by the plan and not shifted to 
a third party through reinsurance. 


Within the HMO, however, there are risk- 
sharing arrangements between the plan and 
physicians.” The extent of “risk sharing” be- 
tween these two decisionmaking entities has 
been tendered as the theoretical explanation of 
observed differences between the performance 
of HMOs and that of other forms of health care 
delivery. But the extent of risk borne by physi- 
cians varies considerably among HMOs, and 
generally but not always, by type of HMO. In 
the IPA, for example, the plan may act essen- 
tially as a third-party insuror: Reimbursed by 
the plan on a fee-for-service basis (according to 
a fee schedule) for services rendered to enroll- 
ees, the IPA physician (and patient) is often 
somewhat more insulated than the PGP physi- 
cian from the financial burden associated with 
decisions regarding the services actually re- 
ceived. 


With respect to hospitalization rates, both 
IPA and PGP enrollees had considerably lower 
rates (25 and 35 percent respectively) than con- 
ventionally insured comparison groups (373). 
These observed differences are due to very dif- 
ferent kinds of decisions being made in these 
HMOs. In the case of the IPA, where physicians 
are often at lower risk for the implicit resource 
allocation decisions that they make, the plan 
translates its own economic incentives and fi- 
nancial constraints to physicians by imposing 
implicit controls on their decisionmaking. Deci- 
sions made at an administrative level to imple- 
ment such mechanisms as preadmission certifi- 
cation and length-of-stay review translate this 


7There are three functional parts to an HMO, which may even 
be legally distinct entities: 1) the plan, 2) physicians (already dis- 
cussed), and 3) the hospital that provides inpatient services (372). 
Usually, the hospital does not enter into risk-sharing agreements 
with the HMO. 
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influence and assure that it will be reflected in 
decisions made at the provider level.® 


Several alternative explanations for the lower 
hospitalization rates of PGP enrollees, and for 
PGP physician behavior in general, have been 
tendered and debated (152,590). One longstand- 
ing, simple explanation has been the availability 
of hospital beds, as determined by explicit re- 
source allocation decisions made by the PGP 
plan (152,590). Such constraints, if rigidly 
adhered to, could be effective controls. Evidence 
of impact on physician decisionmaking, how- 
ever, is minimal (619). Another hypothesis, cur- 
rently supported by the Group Health Associa- 
tion of America (GHAA) and others, is that the 
combination of peer pressures, organizational 
factors, and capitation motivates physicians to 
act in accord with professional incentives as 
well as the constraints of the HMO (590). A 
third hypothesis is that there may be something 
“different,” i.e., in attitudes, values, etc., about 
physicians who choose to practice in an HMO 
rather than in a more traditional fee-for-service 
practice setting. Although this explanation is in- 
tuitively appealing, virtually nothing is known 
about the possible effects of “preselection” on 
physician behavior in the HMO setting (372).° 


The influence of economic incentives and fi- 
nancial constraints on resource allocation deci- 
sions faced by the plan and physicians in the 
HMO may depend on the risk-sharing arrange- 
ments between them. This characteristic varies 
(generally) by type of HMO, but essentially it is 
the plan that is responsible for making admin- 
istrative decisions affecting the financial viabil- 
ity of the organization, while physicians are still 
primarily responsible for decisions affecting the 
well-being of HMO enrollees. The influence of 
physicians, especially heads of services (e.g., a 
chief of surgery), however, is often substantial. 


*The fee schedule used to set reimbursement rates for ambula- 
tory services in the IPA is another example of such direct controls 
imposed on physicians by the plan. Some IPAs have imposed very 
rigid controls on physicians (187). 

*There is also a self-selection process suspected for enrollees 
which is believed to affect choice and disenrollment. Clearly, they 
see the plan as having some advantage, as witnessed by their ten- 
dency to display “voice” (rather than exit) in the face of dissatisfac- 
tion with services (372). 


In general, theory would have it that the 
greater the share of risk borne by each of these 
entities, the greater is the influence of these in- 
centives and constraints on their decisions. With 
respect to explaining the apparent ‘‘cost-con- 
scious” behavior of HMO physicians, this is a 
tenable, but vigorously contested, hypothesis. 
With respect to the plan, however, the criterion 
of risk assures that it is held accountable for 
meeting the conditions fostered by prepayment: 
Its impact on administrative resource allocation 
questions made by the plan is explicit and im- 
mediate. 


The third criterion of a defined population ex- 
erts an influence only insofar as it determines 
the total revenue available to the HMO, and the 
nature of the population for whom it must pro- 
vide services. The defined population of an 
HMO currently ranges from about 3,000 to 2.5 
million enrollees (the latter in five different State 
regions) (372). Predominant age and sex charac- 
teristics of the enrolled population may differ 
markedly between HMOs. The size and charac- 
ter of the HMO population will directly influ- 
ence the types of decisions to be made, as well as 
the administrative structure required to manage 
the organization. But in any case, at any given 
point in time, the HMO can project its annual 
revenue, and enrollees, and therefore estimate 
the demand for various services. Such foresight 
facilitates the achievement of technical efficien- 
cies, i.e., the cost per unit of service, and also 
enables the HMO to make tradeoffs in provid- 
ing benefits and services that will be appropriate 
to its particular population. 


To summarize, the internal economic incen- 
tives and financial constraints faced by the 
HMO derive from the fact that the HMO plan 
has finite resources to allocate, is legally respon- 
sible for meeting the medical needs of its mem- 
ber population, and is also primarily respon- 
sible for maintaining its own fiscal viability. 
This set of conditions, arising from the distinc- 
tive characteristics of prepayment and risk, 
guarantees that the HMO will be “cost con- 
scious.” But the internal economic incentives 
and financial constraints do not necessarily re- 
quire that the HMO be any more (or less) con- 
cerned with efficacy, or effectiveness, of a ben- 
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efit or service in allocating its resources than 
any other health care delivery setting is. 


The Competitive Imperative 


This influence derives chiefly from the char- 
acteristic of voluntary enrollment, which im- 
plies that prospective enrollees have a choice be- 
tween alternative health care providers (and in- 
surors), and that they have the option to disen- 
roll, as well as to enroll, in the HMO (372). The 
HMO, existing in a predominantly fee-for-serv- 
ice environment, must compete for its enrollees: 
Its financial viability depends on its ability to 
generate revenue through enrollment, and con- 
sequently, on its ability to attract enrollees. 


The HMO may compete on the basis of cost, 
as well as the range and quality of benefits and 
services it can offer. In this regard, HMOs have 
a competitive “edge” in the health care market, 
by virtue of their generally much lower out-of- 
pocket costs. Competition can reintroduce the 
concern (by consumers) for cost (300). In the 
situation where HMOs compete against one 
another, the limited data available suggest that 
relatively small differences in premiums can at- 
tract enrollees (372). But when premium rates 
are comparable, the question becomes one of of- 
fering the most (and/or best) services for the 
least money (95). 


The ultimate question for the HMO is what 
impact any change in the services and benefits 
offered will have on the plan premium. Theoret- 
ically, the use of “cost-effective” techniques in 
the development of a comparably attractive, yet 
low-cost, product would seem to be essential to 
attract and retain customers in a competitive 
market. At the least, the competitive imperative 
does not negate the potential for viewing re- 
source allocation decisions in the HMO as 


CEA/CBA problems. 


Ultimately, however, the competitive impera- 
tive means that the HMO must meet the expec- 
tations of enrollees regarding benefits and serv- 
ices, expectations that are dictated largely by 
the norms of practice established by the pre- 
dominating fee-for-service mode of health care 
delivery. The leverage of enrollees to demand 
these services stems from their option to disen- 


roll.1° The competitive imperative implies that 
simply because competing providers are offer- 
ing a service, and doing so in a certain manner, 
the HMO will be induced to provide it also, and 
in a similar fashion. The fact that HMOs must 
also compete in the marketplace for physicians 
reinforces this tendency: HMOs will seek to 
provide opportunities for medical practice com- 
parable to those offered in the fee-for-service 
setting in order to attract and retain physicians. 
This situation will change as the number of 
HMOs increases and they are forced to compete 
among each other as well as with fee-for-service 
care. 


The fact that HMOs must compete for both 
enrollees and physicians means that the HMO 
will try to provide benefits and services similar 
to those available from other insurors and pro- 
viders—but within its financial limits. The 
HMO'’'s overall strategy in making resource allo- 
cation decisions will be to maintain or improve 
its competitive position. This implies that any 
possible efficiencies achieved in providing serv- 
ices, reductions made in the number and/or 
range of services, and/or tradeoffs made in al- 
ternative ways of offering services that might be 
induced by the internal economic incentives and 
financial constraints will be subject to the condi- 
tion that the HMO remain an attractive, com- 
petitive alternative. 


Thus, given both a competitive imperative 
and inherent economic incentives and financial 
constraints, the two questions most commonly 
of immediate concern to the HMO are: 1) how 
to allocate its limited resources to provide the 
given benefits and services in the most efficient 
manner possible, and/or 2) how to reduce inap- 
propriate services and benefits (in volume and 
frequency) without a perceptible loss in quality 
(37a)5 


Evidence appears to indicate that the cost sav- 
ings of HMOs are due to changes in the mix and 
number of services provided. Overall, ambula- 
tory services received by HMO enrollees are no 


10There is a considerable literature regarding what is essentially 


a cost-benefit decision made by consumers to join or opt out of an 
HMO: ”. . . they face the question of whether the benefits they are 
receiving are worth the premiums they are paying” (159). 
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different from those received by persons cov- 
ered under conventional insurance.!! (Among 
other things, this reflects the imperative of the 
HMO to meet the consumer’s demand for ambu- 
latory services.) Rather, as previously indi- 
cated, almost all of the cost savings can be 
directly attributed to the lower (by 30 percent 
overall) hospital admission rates for HMO en- 
rollees (373). 


Similarly, empirical data suggest that persons 
conventionally insured with full coverage for 
preventive ambulatory visits receive at least as 
many, if not more, preventive services than 
HMO enrollees (374). In contrast to the ideolo- 
gy implied in the name “health maintenance or- 
ganization,” the apparent greater use of preven- 
tive services in HMOs appears to be attributable 
to better financial coverage, i.e., the result of 
lower costs of ambulatory visits faced by enroll- 
ees (374). 


Because of its financial incentives and con- 
straints, the HMO (by and large) tends to deliv- 
er a “conservative” brand of medical care. This 
implies that where the cost of a preventive serv- 
ice exceeds that of treatment, and efficacy is un- 
clear, the HMO tends not to provide that serv- 
ice, or to provide less of it (374). With respect to 
preventive services that are of questionable ef- 
ficacy, it will tightly ration the resources re- 
quired to provide them. For example, the North- 
ern California Kaiser Foundation Health Plan 
shifted its stance on health examinations in 1976 
by advocating a triannual rather than an annual 
physical checkup for individuals under a certain 
age. 


Thus, in allocating resources in the HMO, the 
rationing of preventive services appears to be 
one of the principal questions where the poten- 
tial benefits (i.e., efficacy) of a service are con- 
sidered in relation to costs, and to which formal 
CEA/CBA has most often been applied. That 
HMOs should show concern for the efficacy and 


There is considerable variation, however, in ambulatory visit 
rates by type of HMO. IPA model HMOs have a rate almost 20 
percent higher than that of conventionally insured persons; PGP 
model HMOs have slightly varying rates that even out to be only 
negligibly different from the rate for conventional groups. 


effectiveness of preventive (rather than surgical, 
for instance) services in relation to costs may be 
interpreted as another reflection of their adher- 
ence to the status quo. More specifically how- 
ever, it is the low “market medical benefit” of 
preventive services that allows and encourages 
questions regarding their efficacy. 


In summary, the limited empirical evidence 
available regarding HMOs suggests that the per- 
tinent resource allocation questions and deci- 
sions are foremostly influenced by the competi- 
tive imperative, but are also ultimately subject 
to the inherent economic incentives and finan- 
cial constraints in the HMO. 


With the exception of preventive (usually 
secondary) services, the rationing of resources 
in the HMO at an administrative level generally 
does not include the explicit consideration of ef- 
ficacy relative to costs, or of whether or not a 
benefit or service should be offered on that 
basis. Instead, the competitive imperative in- 
troduces the consideration of a “market medical 
benefit,” or the qualitative attribute(s) of a 
benefit or service that will attract and retain 
physicians and enrollees in the marketplace. 
While these are not societal benefits, they are 
assumed to have some health benefit (619). For 
HMOs, these benefits are largely prescribed by 
competitors in the marketplace: Within HMOs, 
they are rationed in terms of how and/or how 
much to provide (rather than whether to pro- 
vide them). 


By and large, the resource allocation ques- 
tions pertinent to this task are concerned with 
achieving technical efficiencies and/or volume 
and frequency processes. These are questions 
generally addressed by CEA/CBAs, though not 
by those that take health effects into explicit ac- 
count. The concurrent appreciation and consid- 
eration of benefits and costs in HMOs may be 
regarded as implicit “net cost’ CEA/CBA, and 
possibly help lead to the “cost-effective” be- 
havior of the organization. The remaining dis- 
cussion in this chapter supports these conclu- 
sions with evidence regarding the use and poten- 
tial usefulness of CEA/CBA in HMO resource 


allocation decisionmaking. 


SURVEY OF HMOs 


Given the limited scope of this investigation, 
and the tremendous diversity possible in organi- 
zations characterized as HMOs, the HMOs in- 
cluded in this small survey do not begin to ap- 
proximate a representative sample. The HMOs 
surveyed here were identified with the assist- 
ance of several individuals who, on the basis of 
their personal knowledge and experience with 
them, suggested various organizations that they 
thought might be most likely to be able to re- 
spond. 


The final selection of 11 HMOs was based on 
whether each HMO had recently made some im- 
portant resource allocation decision, since this 
decision was to provide the initial basis of dis- 
cussion during the interview. Table 5 indicates 
the HMOs interviewed, specifying for each the 
following traits: 1) type of HMO, i.e., staff or 
group model PGP, or IPA; 2) age, i.e., the 
length of time in operation; and 3) size, i.e., the 
number of enrollees. These variables were rec- 
ognized as potentially important influences on 
findings. In the small survey sample, however, 
it was not thought worthwhile to try to select 
HMOs according to these traits. 


Table 5.—HMOs Surveyed: 
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Only one or two individuals were interviewed 
at each HMO, except at Kaiser, where sev- 
eral individuals in research offices were inter- 
viewed.'? Consequently, some of the individuals 
interviewed were analysts, some were adminis- 
trators, and some were physicians. But some in- 
dividuals acted in two or even all three roles. 


There is reason to be skeptical about the gen- 
eralizability of findings regarding the usefulness 
of CEA/CBA in these few HMOs. Because the 
HMOs and individuals interviewed were se- 
lected on the basis of informed (and candid, if 
not objective) opinions as to their ability to re- 


2Special assistance in selecting HMOs and the individuals in 
them was given by Mildred Arrill of OHMO. The suggestions of 
Frank Seubold, Harold Luft, Martha Kendrick, Jane Willems, and 
Morris Collen were also helpful in singling out the several orga- 
nizations and persons surveyed here. 

3In addition to the individuals interviewed in HMOs, almost as 
many persons from OHMO, trade organizations, and scholars of 
HMOs were interviewed. Many of the comments were useful in 
both anticipating the findings and drawing conclusions based on 
them. The comments of these individuals are referenced through- 
out the discussion. Referenced individuals are listed in the bibliog- 
raphy. 


By Type, Age, and Size 

















Name of HMO and location Type Age Size (# enrollees) 

1. Capital Area (Albany) Community Health Plan........ Staff PGP 3 years 27,000 
Latham, N.Y. 

Zanizona MealthiPlan gt. te sos. sort. sie OER ee oR Group PGP 8 years 66,000 
Phoenix, Ariz. 

3. Group Health Cooperative of Puget Sound........... Staff PGP 33 years 270,000 
Seattle, Wash. 

4. Harvard Community Health Plan ................... Staff PGP 11 years 85,000 
Boston, Mass. 

Demo OU rennSyiVaniaws Pele Bis Set Pcen em 5 ade IPA 5 years 42,000 
Willow Grove, Pa. 

6-8. Kaiser Foundation Medical Care Plan@............... Group PGP 47 years 

Northern California Region, San Jose, Calif. ....... 1,637,050 
Southern California Region, San Diego, Calif. ...... 1,554,500 
Oregon Region, Portland, Oreg............-....-. 231,743 

9. Northern Community Health Plan .................. Group PGP 5 years 23,000 
Glenview, III. 

10. Rhode Island Group Health Association............. Group PGP 8 years 30,000 
Providence, R.I. 
11. Rutgers Community Health Plan...............-.-. Staff PGP 4 years 38,000 


New Brunswick, N.J. 








in addition to individuals at these three HMOs, individuals at the Kaiser Permanente Medical Care Program research facility and the Kaiser Foundation Health Services 


Research Center were also interviewed. 
SOURCE: Office of Technology Assessment. 
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spond to the inquiry, however, these findings 
may represent an estimate of the maximum po- 
tential for the use and usefulness of CEA/CBA 


GENERAL FINDINGS 


As previously mentioned, the discussions 
with individuals in the HMOs centered on some 
specific recent resource allocation decision, and 
whether or not CEA/CBA was used, or would 
have been a useful analytical technique in the 
decisionmaking process. The resource alloca- 
tion questions, analytical techniques used, and 
their role in the decisionmaking process were 
ascertained. To ascertain the type of analysis 
used, a brief technical description was obtained. 
Inquiry regarding the usefulness of analysis was 
concerned with determining the decisiveness of 
its impact on the final decision; its effect of a 
political, ethical, economic, or other nature; 
and its contribution to the decision made as 
viewed by the various parties affected. Each of 
these three topics—resource allocation ques- 
tions, analysis used, and the usefulness of the 
analysis—are discussed below with respect to 
both the current and potential appropriateness 
of CEA/CBA in HMO decisionmaking. In addi- 
tion, the extent to which HMOs have institu- 
tionalized the resource allocation process is dis- 
cussed. Finally, the current and potential ap- 
plicability of CEA/CBA to decisionmaking in 
OHMO, and to the HMO statutes that it admin- 


isters, is discussed. 


Current Applicability of CEA/CBA 
in HMOs 


Resource Allocation Questions 


A wide range of resource allocation questions 
were reported in the surveyed HMOs. Most fre- 
quently, HMOs were engaged in a simple “make 
or buy,” “lease or purchase” type of decision. 
The question was not so much whether to do 
something at all, but whether something was 
less costly to do “in” or “out” of the HMO. The 
most commonly cited example of such a deci- 
sion was made with respect to the addition of 
physician staff, usually a specialist (i.e., 


in HMOs. Thus, although these findings should 
be regarded as very preliminary, they are not 
without significance. 


whether to contract with a physician in the com- 
munity, or to hire another physician on staff of 
the HMO). Other examples of “lease or buy” 
types of decisions were made with respect to ob- 
taining: 1) improved transcription (of medical 
records) services; 2) diagnostic ultrasound 
equipment; and 3) new physical facilities. The 
“make or buy” decisions arise because of the 
capital expenditure issue associated with them, 
but more importantly, they arise from the im- 
perative exemplified in these statements: 


Every time a new technology comes into 
medical practice, we groan because we have to 
find some way of getting it . . .(527); 


If the “Docs” come to us and say they want to 
do open heart surgery, the question we face is 
not whether, but how we can do it (369); and 


We are conservative, practicing “spartan” 
medicine, (and) perceived of as “tightwads,” but 
we are also constrained to go along with what is 
practiced in the medical community... If 
someone introduces something new, then we 
have no choice but to get it or do it somehow 
(619). 


A common resource allocation question in the 
HMO, therefore, is “How can we provide what 
is demanded?” 


Other resource allocation decisions reported 
in the HMOs, however, tended to be more con- 
cerned with whether or not a particular service 
should or could be provided. Some of these de- 
cisions were whether to: 1) implement the 
COSTAR (medical information) system rather 
than some alternative systems, 2) install a cen- 
tralized v. a decentralized automated reception 
system, 3) cease doing intestinal bypass surgery 
on obese patients, 4) implement a health educa- 
tion program (2 cases), and 5) offer biofeedback 
as a palliative care measure. 


Even in the case of the two-option “make or 
buy” type of decision, where the benefits or ef- 
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fects are generally accepted as being similar, the 
concern is not solely for efficiency. “Relative to 
just saving money, there is (at least) the assump- 
tion that what’s being done is of benefit. . .,” 
either to the organization (in the case of admin- 
istrative decisions), or to enrollees (in the case of 
medical services); the benefit is assumed (619). 
Often, questions about benefits were identified 
on the basis of patient or physician dissatisfac- 
tion with current practices and the desire for im- 
proved quality rather than a less expensive way 
of providing the service. 


In the case of questions involving the aspect 
of whether to provide a service, however, the 
relative benefits and effects of various alterna- 
tives were explicitly compared, as in the case of 
COSTAR and the automated reception system 
decisions. In the case of one of the health educa- 
tion programs, the decision involved evaluating 
the efficacy associated with these services. In 
particular, planners were specifically asked to 
measure the health benefits, i.e., how is this 
service going to improve health? In order to an- 
swer the question of “should we spend those 
dollars?” the health benefits (and to whom they 
pertained) had to be proven. In the case of bio- 
feedback, a clinical trial of that technique was 
being undertaken to ascertain whether it should 
be made available. 


In all questions, the associated parameters 
were limited by the rigorous constraints of time 
and money. Consequently, much of the re- 
source allocation process in HMOs could be 
characterized as “problem-solving,” in which 
problems were often identified by those closest 
to them. For example, at an administrative lev- 
el, there is a great deal of monitoring—of hospi- 
tal admissions and length of stay, referral pat- 
terns among physicians, and so on. In the case 
of intestinal bypass surgery, the problem was 
flagged by the observation that “too many” 
complications were developing in postsurgical 
obese patients. In the case of the health educa- 
tion program, it was noted that the HMO was 
losing its “healthier” enrollees, because “sicker” 
enrollees impeded their access to HMO services. 
In that case, it was thought that a health educa- 
tion program might increase enrollees’ satisfac- 
tion and thus help retain the “heaithier’’ ones. 


Another “problem solving” example is found in 
the implementation of a health appraisal sys- 
tem. In that case, the problem was recognized, 
analyzed, and resolved by the physician in 
charge within the medical department. The phy- 
sician in charge faced a huge backlog of initial 
physical examinations due to recent rapid ex- 
pansion of the HMO and limited physician 
manpower. On the basis of previous research 
and principles of multiphasic health testing 
(106), he devised a “linear system” for “batch- 
ing” intake examinations and freed physician 
time by substituting other health personnel in- 
sofar as possible. 


Thus, much of the resource allocation occur- 
ring in HMOs may be characterized as incre- 
mental decisionmaking where the parameters 
are subject to the constraints of cost and 
timeliness of a solution given the existing situa- 
tion and possibilities: The number of available 
options is usually very small. Even at an ad- 
ministrative level, there are few “strategic” 
resource allocation questions. There appears to 
be very little “zero-based” decisionmaking in 
HMOs. The HMO rarely faces the question of 
whether to decentralize or centralize (e.g., to 
have two or six physical facilities). Instead, it is 
faced with the question of where to build the 
next facility. 


At an administrative level, HMOs are simply 
trying to be more efficient, an objective which is 
not so different from that of other organization- 
al entities delivering health care (369). ‘Cost ef- 
fectiveness,”” however, is viewed as being an in- 
tegral consideration in the decisionmaking proc- 
ess at all times. An interviewee’s perception of 
whether the resource allocation questions ad- 
dressed by HMOs were questions of cost effec- 
tiveness depended greatly on whether the in- 
dividual was familiar with the formal technique 
of CEA/CBA or instead had a more “lay” sense 
of the meaning of “cost effectiveness” (such as 
that exemplified in the statement “we must be 
cost effective’). In the former case, the resource 
allocation questions were generally not viewed 
as being amenable to sophisticated, health- 
effects-oriented CEA/CBA, but in the latter 
case, they were viewed as problems of cost ef- 
fectiveness. 
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Analyses Used 


Generally speaking, formal CEA/CBA be- 
yond “net cost” analysis was not found to be 
used by HMOs. This does not imply, however, 
that benefits were not integrated into their anal- 
yses. It was recognized that it is possible to be 
efficient without being effective. A common 
perception was that in considering every major 
decision, there would be an evaluation as to 
whether the service would be “worthwhile.” The 
criteria of both cost and quality were included 
in making this determination: The first question 
was often, “how will this affect the quality of 
service?” and the second, “what is the impact on 
the cost per member per month?” Analyses, 
therefore, involved the evaluation of both costs 
and benefits: Analyses were characterized by 
the objective of ascertaining the financial impact 
of a given (or multiple) option(s), but the idea of 
quality loomed over all considerations. 


Two examples illustrate this generalization re- 
garding analyses used in HMO decisionmaking. 
The example of whether to lease or purchase ul- 
trasound equipment illustrates that benefits 
were also taken into consideration in the costing 
out of options, and that the question of medical 
efficacy of diagnostic ultrasound also arose in 
considering those benefits. Questions surround- 
ing this “cost” analysis included not only what 
the costs of the lease option would be, but also 
whether the equipment could be put to other 
uses (besides obstetrics) in the HMO. This ques- 
tion involved ascertaining whether ultrasound 
diagnostics could replace other diagnostic tests, 
for example, CT scans in some cases, and the 
costing out of such “replacement” effects. Radi- 
ologists and internists in the HMO were ques- 
tioned on the potential benefits of these other 
uses of ultrasound. These physicians concluded 
that the use of ultrasound in these capacities 
would still have to be backed up by other diag- 
nostic tests. Consequently, the decision was 
made that it was not “worthwhile enough” to 
purchase the equipment at that time, and the 
lease option was chosen. 


The second example of the level of detail at- 
tained in evaluating the costs and benefits asso- 
ciated with a particular decision is the case of 


the COSTAR medical information system. Al- 
though, it was clear from the outset that the 
COSTAR system would probably be a more ex- 
pensive option, it was also recognized that it 
could provide a completely different level of 
service in the HMO. An extremely detailed 
analysis of per dollar-impact over the course of 
several years was conducted. Part of this anal- 
ysis involved estimating how far into the future 
the various options would work and included 
such factors as the rate of growth (of the HMO) 
and the future addition of a (third and fourth) 
medical center. Qualitative aspects of “after 
hours” accessibility and the continuity of care 
were also crucial factors. Even potential prob- 
lems of future energy shortages and how they 
could affect an automated recordkeeping system 
were considered. Though in the long run, a pro- 
jected yearly cost difference of $20,000 to 
$40,000 more for COSTAR v. other systems 
was estimated, COSTAR was chosen on the ba- 
sis of its clear-cut superiority, i.e., its “different” 
level of service. 


A wide range of analytical expertise and re- 
sources was found to be applied to the resource 
allocation questions in diffferent HMOs. Prac- 
ticing physicians, economists, operations re- 
searchers, and accountants in various adminis- 
trative and/or research capacities were found to 
be “analysts:” Even in small HMOs, where the 
organization is run very “lean” and there are 
few spare resources to devote to analyses used 
in decisionmaking, analyses were being con- 
ducted and were generally more sensitive to the 
costs and benefits of various decisions than 
might have been expected. Where resources al- 
lowed, as in the case of a health education pro- 
gram, methodological expertise was sought to 
deal with the question of determining the asso- 
ciated health benefits. Only the very largest 
HMO was found to have conducted a formal 
CEA/CBA “in-house” (and this with some Fed- 
eral support). By and large, the “in-house” ef- 
forts and resources applied in pursuit of these 
questions were a function of the size, age, and 
financial situation of the particular HMO. 
Though HMOs were generally thought of as be- 
ing “too small” to do CEA/CBAs, it is clear that 
none of the HMOs in this limited survey were 
too small to conduct analyses that were respon- 


sive to the perceived information needs of the 
problem at hand. 


The analyses themselves ranged from “back- 
of-the-envelope” to rigorous fiscal analyses. The 
consensus opinion was that most resource allo- 
cation questions relevant to the HMO did not 
require formal, sophisticated CEA/CBA to as- 
certain a satisfactory answer because the param- 
eters of the decisions were relatively narrow. In 
the case of developing and implementing a 
health appraisal system, the physician in charge 
did a “back-of-the-envelope cost analysis,” in 
the belief that if cost savings (in terms of substi- 
tion of physician for other medical manpower) 
were not evident at that crude level, they would 
probably not show themselves in a more de- 
tailed version of analysis. 


It was commonly appreciated that there was 
never enough time (or money) to do all the anal- 
ysis one might wish, and that decisions had to 
be, and ultimately are, made on the basis of the 
information at hand at the time the decision is 
made. Concomitantly, however, there was an 
overall confidence in HMO staff’s comprehen- 
sion of the scope of most resource allocation 
questions and in the extent and depth of in- 
formation required to get an answer that was 
satisfactory, if not absolutely optimal. 


The OTA survey examined whether the 
HMOs borrowed CEA/CBA information from 
external sources if the HMO’s own resources 
were not sufficient to produce such information 
internally. The HMOs reportedly consulted the 
medical CEA/CBA literature, but relatively in- 
frequently used it as a basis for decisions. The 
decisions in which it was used as a basis (all 
made by a single HMO) included the elimina- 
tion of routine chest X-rays, modification of 
adult physicial examinations and pediatric pre- 
vention schedules, and indications for CT scans. 
In other cases, including electronic fetal moni- 
toring (EFM) and coronary bypass surgery for 
angina, the literature was also considered, but 
was regarded as too equivocal to be used as the 
basis of deciding to withhold these services. 


Thus, while being “cost effective’ was viewed 
as essential, the rigorous forms of CEA/CBA 
were seen as being largely irrelevant in these 
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HMOs. Rigorous CEA/CBA studies, among the 
HMOs surveyed, were found to be very rarely 
required by resource allocation questions, even 
more rarely conducted, and only infrequently 
borrowed and applied. 


Role in Decisionmaking 


A general consensus on the role of analysis in 
the resource allocation decisions found in this 
survey is summarized by the following state- 
ment: “The analyses are critical, . . . (we) 
couldn't make the decisions without them, but 
the analyses do not make the decisions. . .” 
(330). Thus, analyses were widely appreciated 
as an integral part of the decisionmaking proc- 
ess in HMOs, even though that process is still 
perceived as an essentially political one. Impor- 
tant functions of these analyses were the follow- 
ing: 1) to define the parameters of the decision 
at hand, 2) to clarify the impacts of various op- 
tions, and 3) to serve as a basis for the discus- 
sion of issues associated with a particular deci- 
sion. 


These generalizations are exemplified in the 
case of COSTAR. The main impact of that anal- 
ysis was said to be that it was“. . . the basis for 
bringing all the issues forward in a political 
power struggle . . .” (568). 


The aforementioned observations suggest 
several conclusions regarding the usefulness of 
analyses in the decisionmaking process. First, 
they indicate that when introduced into the 
“political” decisionmaking environment, any 
“objective” analysis will come under harsh 
scrutiny by those representing the opposing 
viewpoint. They also suggest the constraints of 
time and resources within which analysts and 
HMO managers and other decisionmakers must 
operate, i.e., how few options present them- 
selves, and how many fewer still may be ana- 
lyzed and considered. They also suggest that 
among these alternatives, the option about 
which analysts and decisionmakers may have 
the most information, or with which they are 
most familiar, may have a greater likelihood of 
being chosen: The decision made will often be 
the one where the most detailed information is 
available. While this may not be the “best” or 
optimal choice, it is a rational one. 
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Many of the cost analyses might technically 
be termed ‘net cost” analyses, in one sense an 
extreme of CEA/CBA. Yet when these analyses 
were incorporated into the decisionmaking 
process, decisionmakers became extremely sen- 
sitive to the aspects of quality and benefits: 
Though it was not considered to be the most 
“cost-effective” option, i.e., in the sense of least 
cost, the COSTAR option was chosen anyway 
on the basis of its clearly superior capabilities 
(as well as power politics). 


The COSTAR example illustrates that per- 
haps the first reason to chose a particular option 
is so-called “program effectiveness.’” At a sub- 
liminal level, it is physician and enrollee satis- 
faction which is the benefit of foremost concern. 
There is great hesitance to do anything that does 
not have readily discernible benefit, and any- 
thing without an immediately perceived value 
(however defined) will probably not be done at 
all. 


We have a long history of taking care of the 
medical needs of people. It is very difficult to 
justify withholding something on the basis of 
analyses in general, and on the criterion of costs 
in particular . . . What people want is what is 
important . . . (527) 


Institutionalization of 
Decisionmaking Processes 


The degree to which resource allocation deci- 
sionmaking and the analyses were formalized, 
or institutionalized, is (to some extent) a func- 
tion of the size and financial situation of the 
HMO, and consequently, of the administrative 
structure required to manage it. The majority of 
examples of resource allocation decisions dis- 
cussed so far may be characterized as ‘problem 
solving,” where those individuals closest to the 
problem have identified and possibly even ana- 
lyzed and resolved the problem. In general, 
these persons were not “handed” solutions (i.e., 
resource allocation decisions) by managers from 
higher echelons of the administrative hierarchy. 
Such “problem solving” was critical to the effi- 
cient allocation of resources within particular 
HMOs, and is a standard, ongoing feature, even 
if it is not a formalized procedure. 


Institutionalized, formal analysis for resource 
allocation decisionmaking was the rare excep- 
tion which proved this rule. The decisionmak- 
ing process meeting these criteria that was most 
often cited was the “budgetary process.” It was 
often perceived that CEA/CBA takes its most 
explicit form in the HMO in the annual budget- 
setting process. In HMOs, both large and small, 
the budgetary process involved projecting the 
annual revenue (based on forecasts of member- 
ship), and estimating the budget (based on esti- 
mates of demand for services by that popula- 
tion). 


In two of the largest and oldest HMOs, Kaiser 
Foundation Medical Care Program (KFMCP) 
and Group Health Cooperative of Puget Sound 
(GHC), examples of very rigorous, institution- 
alized procedures for allocating resources were 
found. Without delving into the complex orga- 
nizational and administrative structure of 
Kaiser, it is sufficient to say that: 1) there is an 
overall vertical hierarchy within both the over- 
all organization and in each of the five regions, 
and 2) that the program has established consid- 
erable (relative to any other HMO) research ca- 
pabilities and “in-house” analytical expertise. In 
both the KFMCP Central Office and in the 
Southern California Regional Office, there are 
what are called “benefit-cost offices.” These of- 
fices essentially perform financial impact anal- 
yses, i.e., they examine the impact of a pro- 
posed change in resource allocation on the cost 
per member per month. According to the direc- 
tor of one of them: “... We do not do cost- 
benefit analysis here—at least like they are done 
in academic or government institutions... 
where the bottom line is life-years saved, or 
some other society-based measure...” (527). 
Analysis of both costs and benefits is based on 
the internal economics of KFMCP. Thus, Kaiser 
does appear to conduct some “program-perspec- 
tive’ CEA/CBAs. 


At least in the Northern California Office, re- 
quests for analyses are approached in a stand- 
ardized fashion, with preprinted worksheets be- 
ing used to evaluate various proposed changes. 
Discounting (usually at the rate of 10 percent) 
and sensitivity analysis are a standard part of 
the evaluation. The analyses typically examine 


outcome measures such as total net benefit and 
internal rate of return on investment, as well as 
a cost-benefit ratio. They are key planning tools 
in daily use at Kaiser, and results are regarded 
as proprietary information. Both the perform- 
ance and use of these analytical tools have been 
incorporated in the organizational structure and 
administrative processes of KFMCP. 


Two other examples of the institutionaliza- 
tion of research capabilities and analytical ex- 
pertise in Kaiser are found in: 1) the Division of 
Technology Assessment in Oakland, Calif.;14 
and 2) the Health Services Research Center of 
the Bess Kaiser Foundation in Portland, Oreg. 
At the former, the assessment of the efficacy of 
biofeedback as a palliative care measure for var- 
ious conditions is an example of the research 
conducted in the relatively new analytical enti- 
ty. This research will be one input, along with 
“|... organizational impact, long-term cost 
considerations, and patient satisfaction, as well 
as other alternative measures...” in making 
the decision of whether or not to include bio- 
feedback as a benefit (300). 


The Health Services Research Center has pub- 
lished several studies on the “cost effectiveness” 
of the substitution of nurse practitioners and/or 
physicians assistants for physician manpower in 
the HMO setting (496,497). Research on “do- 
not-admit” surgery was also conducted at this 
center (263).15 


The results of both have been used through- 
out KFMCP (as well as other HMOs) resulting 
in efficiencies and thus, cost savings. Interest- 
ingly, although 


. . . there was never a conscious decision made 
(by management) to pursue a “do-not-admit” 
strategy, there were soon 20 percent of all sur- 
geries being done on a “do-not-admit” basis. As 
the idea “caught on,” and administrative sup- 
port developed, 40 percent of surgery was soon 
being done in this way . . . . The cost implica- 
tions of this change (in physician behavior) were 
enormous. . . (263) 


4This new Division was created in the summer of 1979, and is 
currently headed by Dr. Morris F. Collen. 

18Do-not-admit” surgery refers to cases in which relatively sim- 
ple surgical procedures are performed in the operating room, but 
the patient is not admitted unless there are complications (263). 
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This and the nurse practitioner/physician assist- 
ant example illustrate the use of formal meth- 
odologies to evaluate various configurations of 
resources, but the analyses were only loosely, 
and sometimes not at all, related to “across the 
board” policy decisions made by management. 


At GHC in Seattle, the Pharmaceutical and 
Therapeutics Committee is a specific example of 
the institutionalization of cost-effectiveness and 
cost-benefit considerations in organizational de- 
cisionmaking. Prescription drugs, included in 
the benefits package of GHC enrollees, are sub- 
ject to review and approval by this committee: 
The objective is to regulate the availability of 
prescription drugs (within the plan). No GHC 
physician may introduce a pharmaceutical into 
the “stockpile” of those already available unless 
it has been passed by this committee. The proc- 
ess is similar to those of formularies in hospi- 
tals. For example, with the appearance of a 
“new” antibiotic on the market, an ‘‘across-the- 
board” decision will be made to use either the 
new or the old drug, but not both, in that HMO. 


In considering the addition of another drug, 
some of the questions asked by the committee 
were (given in this order): 1) Is it effective? 2) Is 
it costly? 3) Are there other, and perhaps too 
many other, options already in. stock? and 
4) What is its effectiveness and cost relative to 
these options? Data for these decisions were 
taken from the current available literature, 
clinical experience, and so on. It was claimed 
that this control resulted in very substantial (50 
percent) reductions in the cost per (comparable) 
prescription for GHC enrollees. The committee 
is an example of a formal review mechanism in 
one HMO that has institutionalized the consid- 
eration of “cost effectiveness” as a decision 
criteria—and specifically the medical (or health) 
benefit—if not the actual conduct of formal 
analyses. 


Potential Applicability of 
CEA/CBA in HMOs 


Resource Allocation Questions 


The question of whether something is cost ef- 
fective or cost beneficial—in the most formal 
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analytic sense of the term—is by and large an 
academic one in HMOs. As perceived by the Di- 
rector of the Kaiser Health Services Research 
Center (262): 


... posing the question inherent in these 
analyses is like asking “. . . if the money supply 
is tightened in our economy, then. . .” These 
analyses are too broad and inclusive, while the 
problems of management are much more con- 
strained: CEA/CBA do not fit the parameters of 
management decisions. 


Another criticism of CEA/CBA voiced by 
both analysts and decisionmakers was that, 
besides insufficient time and money, there was 
inadequate freedom to ask such questions 
(619): 


We must serve the same psychological and so- 
cial, as well as medical, needs that traditional 
providers do . . . we must be able to react to the 
same emotional and social pressures . . . (even 
though) we think in per capita terms. . 


There are two reasons why “. . . it would be 
very difficult to justify withholding a service on 
the basis of a CEA/CBA .. .” (527). First, as in 
the fee-for-service practice setting “. . . an ethi- 
cal/medical question intervenes in trying to use 
CEA/CBA as a basis for making resource 
allocation decisions . . .”” (588). Secondly, the 
HMO faces competitive pressure to offer as sim- 
ilar (in appearance) services as possible to those 
of fee-for-service providers in order to meet the 
expectations of both physicians and enrollees re- 
garding comprehensiveness and quality of serv- 
ice. 


Decisions involving ethical and moral ques- 
tions, as well as those that are otherwise value 
laden, are perceived as “risky” areas for the rig- 
orous application of analysis such as 
CEA/CBA. For example (619): 


. . although the availability of hospital beds 
may be held at a very finely tuned ratio of 1.5 or 
1.6 beds per 1,000 enrollees, when there is a 
“crunch,” then (we) physicians simply seek other 
available hospial beds in the area . . . we don’t 
not admit patients who in our judgment require 
hospitalization . . . 


Similarly, the implementation of a midwife pro- 
gram initiated by enrollees would not be a deci- 


sion likely to be subjected to, or influenced by, a 
formal analysis. 


Decisionmakers will not be inclined to rely on 
analysis in addressing resource allocation ques- 
tions where analysis could too easily (because of 
uncertainties) lead to the “wrong” answers. 


Generally speaking, the question of whether 
something is cost effective or cost beneficial in 
the absolute sense is seen appropriately as hav- 
ing little overall practical utility in the HMO. 
This view was expressed by two high-ranking 
officials of GHAA: 


. . . There is a misplaced emphasis on the “ab- 
solute” cost-effectiveness of capital intensive in- 
vestment like lab analyzers, or CAT scanners, or 
open heart surgery facilities when the more im- 
portant question is how (at what level or intensi- 
ty) these are utilized . . . (588); 


and 


. . . Much of the cost-effectiveness literature 
is around decisions on individual technologies, 
but it is the decisions regarding volume and fre- 
quency processes (such as laboratory and other 
diagnostic testing) where there is the greatest po- 
tential for cost savings . . . (299). 


HMOs seem to be more straightforwardly 
concerned with returns to scale, returns on in- 
vestments, etc. Thus, in HMOs, “cost effective- 
ness” is correctly interpreted as a relative, rather 
than an absolute, attribute: There may be many 
“levels” of “cost effectiveness” associated with a 
particular option rather than an answer to the 
binary question of whether or not an option is 
“cost effective.” 


Analyses and Role in Decisionmaking 


HMOs’ “in-house” research and analytic ca- 
pabilities were usually found to be too limited 
(in terms of both money and expertise) to allow 
the conduct of rigorous and broad-based CEA/ 
CBAs or clinical trials of efficacy. Other poten- 
tial ways of obtaining CEA/CBA information 
are through technical assistance and the existing 


CEA/CBA literature. 


Technical assistance by which CEA/CBA 
might be obtained could potentially come from 
three sources: 1) private consulting firms; 


2) GHAA, the trade association for prepaid 
group practice type HMOs; and 3) OHMO in 
DHHS. The financial resources required to ob- 
tain this from private consulting firms would 
make this source an infeasible one for many 
(probably most) HMOs, even if they were inter- 
ested in pursuing a CEA/CBA problem. OHMO 
concedes that there is no reason why CEA/CBA 
could not be done under the existing statutory 
provision for general technical assistance to 
HMOs (159,313). Although it is conceivable 
that a CEA/CBA problem could be undertaken 
within this technical assistance provision, the 
possibility seems remote in light of competing, 
and more pressing, problems. Finally, GHAA 
also brokers and provides technical assistance to 
its member HMOs. GHAA’s Associate Director 
confidently stated, however (588): 


... 1 would expect that the chances of an 
HMO’s approaching us to do a CEA/CBA are 
next to nothing . . . because these questions just 
aren't there... 


Although several instances were found where 
the available CEA/CBA literature was con- 
sulted regarding the specific technologies, this 
resource was more often regarded as equivo- 
cal—both because of the technical difficulties 
and uncertainties of analyses, and the lack of 
available efficacy information (619): 


.. . (we) have looked to the literature for a 
consensus on EFM, . . . if there were decisive 
evidence, then we could use that to make deci- 
sions regarding its use... but there’s not 

. (we) wish there were, but there isn’t. . . 


The issues of transferability of existing CEA/ 
CBAs to individual HMOs in particular, and to 
HMOs in general, adds yet an additional source 
of uncertainty (295): 


... We just don’t have the health delivery 
knowledge (in HMOs) yet to be able to predict 
“what happens if ’X’ happens?” . . . even in one 
HMO, let alone to generalize. . . 


All things considered, a representative consen- 
sus would seem to be that ” . . . various uncer- 
tainties make formal analyses available in the 
literature an inadequate basis for decisionmak- 
ing’. s + (615); 
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Thus it seems that even if CEA/CBA were di- 
rectly relevant to the resource allocation proc- 
esses in HMOs, most HMOs would be hard- 
pressed either to produce or even to find and 
borrow CEA/CBA information that would be 
useful to their individual needs. 


The receptivity of decisionmakers to analyses 
is summarized in the following comments: 


. .. when a CEA/CBA will tell you to throw 
out a certain option, and you intuitively feel that 
it’s the right one . . . usually you disregard the 
analysis . . . but in any case. . . you scrutinize 
and challenge the analysis . . . (568); 


. . . Decisionmakers know or will find the 
limits of analytic tools, even when analysts 
don’t, because if the analysis is to be used as a 
basis for decisionmaking . . . and if the analysis 
turns out to be wrong, it is the manager who 
must endure the consequences . . . (295). 


Given the motivation—political, ethical, eco- 
nomic, or other—to do so, someone can cite 
enough technical problems to make any analysis 
equivocal. Recommendations for changes in re- 
source allocation based on analysis, therefore, 
have been and are likely to continue to be tenta- 
tive, with pilot and demostration projects and 
considerable nonanalytic input preferred as the 
basis for decisionmaking (295). 


There also seem to be misgivings about the 
“overall post facto” nature of research general- 
ly, and about analysis that tends to follow 
rather than precede decisions. As noted by one 
observer (588): 


...CEA/CBA are in “vogue” again... 
When I worked for CDC (Center for Disease 
Control) years ago, we used CBA to evaluate 
tuberculosis vaccination programs, and I can tell 
you that even then, the answer came before the 
analysis was ever done, and it always justified 
the decision. . . 


Institutionalization of 
Decisionmaking Processes 


Although it was found that the conduct of 
analyses and its incorporation in HMO deci- 
sionmaking was typical of the HMOs surveyed, 
it was also found that there was considerable 
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variation among HMOs with respect to whether 
and how these processes were formalized. An 
additional finding was that even though sophis- 
ticated forms of CEA/CBA were not perceived 
as relevant, the issues perceived to underlie 
CEA/CBA were considered an essential part of 
the decisionmaking process at all times. 


More specifically, however, if the most po- 
tentially pertinent question in the HMO is “at 
what level of provision is ‘X’ cost-effective?” 
then the institutionalization of particular orga- 
nizational procedures such as peer review of 
patterns of use of various resources, what con- 
stitutes inappropriate use, and who is qualified 
to designate their use in the HMO, as well as the 
monitoring of referral patterns, hospital admis- 
sions, and the like, are potentially very useful 
institutionalized mechanisms for assuring cost 
effectiveness in HMOs. Many HMOs already 
have instituted some of these formalities, and as 
a result have achieved dramatic cost savings 
(344). 


Because of the great diversity of HMOs and 
variation in the competitive positions they hold 
in their respective communities, however, it is 
important that these procedures remain “‘indi- 
vidualized” (rather than standardized), i.e., 
each according to the HMO’s need for analyses 
and/or review (295). Recently, GHAA has rec- 
ommended to the council of the National Center 
for Health Care Technology that a study be un- 
dertaken with the purpose of developing criteria 
for the acquisition of technological resources in 
HMOs. The proposed study would indicate the 
probable impact of various acquisitions on the 
costs and utilization of these technologies and 
their relationship to per capita and aggregate 
costs, as well as the applicability of findings to 
other clinical organizational settings (300). The 
challenge of this proposal is formidable indeed. 
Still, the results might be useful if posed as a 
relative guide or as one model to follow. 


Statutory Requirements for 
CEA/CBA in HMOs 


Until January 1, 1980, the regulations pro- 
mulgated under the Health Planning and Re- 
sources Development Act of 1974 (Public Law 
93-641) required that HMOs demonstrate the 


cost effectiveness of their requests to State 
health planning agencies for CON approvals of 
capital expenditures. During 1979, an extended 
process of developing a detailed set of criteria to 
be used to ascertain the “cost effectiveness” of a 
given proposal raged on. The 1979 amendments 
(Public Law 96-79) to the health planning law, 
however, exempted HMOs from CON review 
by local HSAs and approval by State planning 
agencies. Neither Federal nor non-Federal 
HMOs are now covered under CON regula- 
tions. Because of the new health planning law, 
the controversial endeavor regarding cost-effec- 
tiveness criteria for HMO proposals was sus- 
pended. In January 1980, the relationship and 
responsibilities of HMOs to local HSAs and 
State health planning agencies were described as 
being in a state of limbo (159). 


OHMO has a formalized system to assist po- 
tential or young HMOs in making cost-effective 
resource allocation decisions. HMOs are per- 
ceived as businesses by OHMO, and the main 
talent and skills required are cited as those of 
management. Therefore, under the authority of 
the technical assistance provisions of the Health 
Maintenance Organizations Act, OHMO has 
developed and published six manuals or review 
guides addressing the topics of: finance, quality 
assurance and health services delivery, manage- 
ment information systems, marketing, structur- 
al and contractual relationships, and manage- 
ment arrangement assessment (308). 


To aid applicant HMOs in making cost-effec- 
tive decisions, the review criteria provide cost 
ranges and reasonable upper cost limits at vari- 
ous levels of enrollment with which quoted bids 
may be compared. Similarly, in the quality as- 
surance and health services delivery review 
guide, a table indicates the number of physi- 
cians required, as well as staffing pattern sug- 
gestions, at various levels of enrollment. (These 
standardized criteria for cost-effective perform- 
ance are intended to serve as benchmarks only, 
and are not meant to indicate mandatory stand- 
ards.) In summary, it seems that the analyses, 
decisionmaking practices, and criteria and 
standards formally institutionalized in the ad- 
ministrative procedures developed by OHMO 
are very similar in spirit to those less formalized 
in, but typical of, individual HMOs. 


Appendixes 


Appendix A: Method of the Study 

Appendix B: Survey of Agency Use of CEA/CBA 

Appendix C: Survey of the Resource Costs of CEA/CBA 

Appendix D: Values, Ethics, and CBA in Health Care 

Appendix E: Legal Status of CEA/CBA 

Appendix F: Description of Other Volumes of the Assessment 

Appendix G: Acronyms and Glossary 

Appendix H: Health Program Advisory Committee, Authors of 
Case Studies of Medical Technologies (Background 
Papers #2, #3, and #5), and Acknowledgments 


Contents 


Appendix-A;. Method of the Study Vie. Sr oe ss ae ee 
‘Appendix B: Survey of Agency: Use of CEA/CBA. . ovae a 
Appendix C: Survey of the Resource Costs of CEA/CBA .............----45- 
Appendix D: Values, Ethics, and CBA in Health Care ...................... 
Appendix Er Legal Status of CEA CBA oo. a ee en 
Appendix F: Description of Other Volumes of the Assessment ................ 
Appendix G: Acronyms and Glossary) 70.600 ea ee 
Appendix H: Health Program Advisory Committee, Authors of Case Studies 

of Medical Technologies (Background Papers #2, #3, and #5), 

and Acknowledgments 25.05 3% b.s6a, Voc ee a eee es Oe 


rT ee 


Appendix A.— Method of the Study 





Most of the studies undertaken at OTA rely on the 
advice and assistance of an advisory panel of experts. 
The advisory panel for a particular assessment sug- 
gests source materials, subject areas, and perspec- 
tives to consider; assists in interpreting information 
and points of view that are assembled by OTA staff; 
and suggests possible conclusions based on the accu- 
mulation of information produced by the study. The 
panel members review staff and contract materials 
for accuracy and validity, discuss policy options of 
the study, and present arguments for and against the 
options and conclusions. They do not determine the 
report's final form, however, and are not responsible 
for its content, direction, or conclusions. 

The advisory panel for the current assessment con- 
sisted of 12 experts with backgrounds in business, 
ethics, health policy, law, economics, statistics, and 
medicine. The panel was chaired by Dr. John R. Hog- 
ness, former President of the University of Washing- 
ton and current President of the Association of Aca- 
demic Health Centers. Two members of the OTA 
Health Program Advisory Committee, Dr. Stuart H. 
Altman and Dr. Frederick Mosteller, also served on 
the advisory panel. 

The first panel meeting was held in Washington, 
D.C. (the site of all four panel meetings), on Decem- 
ber 13, 1978. Panel members reviewed the study plan 
of the assessment, examined the need for specific case 
studies, and reviewed a number of suggestions for 
potential case study examinations. The case study 
approach, as part of the general study framework, is 
intended to illustrate the uses of CEA or CBA in 
health care, especially to evaluate medical technol- 
ogy, along with the impacts, and strengths, and 
weaknesses of these uses. The advisory panel was in- 
strumental in helping the staff set goals for the assess- 
ment, establish boundaries for its focus, and define 
the basis for and role of CEA or CBA as a decision- 
assisting tool in the health care system. The panel 
was also helpful in identifying public and private sec- 
tor uses and users of CEA/CBA techniques or infor- 
mation in decisionmaking. 

To help select medical technologies for the final list 
of case studies, the following criteria were devel- 
oped: 

¢ examples of types of technology by function 

(preventive, diagnostic, therapeutic, and reha- 
bilitative); 

¢ examples of different stages of development and 

diffusion (not yet diffused, experimental or pi- 
lot, established in medical care); 


¢ examples from different areas of medicine (such 
as general medical practice, pediatrics, obstet- 
rics, radiology, and surgery); 

examples addressing medical problems that are 
important because of their high frequency or sig- 
nificant impacts; 

examples of technologies with associated high 
costs; 

® examples of technologies in widespread use; and 

© examples with sufficient evaluable literature. 

On the basis of these criteria and panel recommen- 
dations, OTA staff selected the case study topics. In 
addition, the Senate Finance Committee had specifi- 
cally requested four case studies: psychotherapy, res- 
piratory therapy, diagnostic X-ray, and length-of- 
stay in hospitals. Nineteen case studies (identified in 
app. F) were added to the study plan (Background 
Papers #2, #3, and #5). 

The second advisory panel meeting was held on 
February 14, 1979. The panel reviewed the literature 
gathered by OTA staff, suggested additional refer- 
ences and sources of information, and evaluated the 
case study plan. In addition, it examined the staff's 
preliminary work on CEA and CBA methodology to 
identify strengths, weaknesses, and omissions. The 
panel was also helpful in commenting on the decision 
area papers (e.g., use of CEA in reimbursement 
coverage decisions) that were being prepared by the 
staff. Finally, it was asked to review a separate study 
being conducted as part of the overall assessment: 
The Management of Health Care Technology in Ten 
Countries (Background Paper #4). 

Several subprojects were pursued: a survey of rele- 
vant State, Federal, local, and private and nonprofit 
agencies and organizations to determine the extent of 
use or support of CEA/CBA activities (app. B); and 
an extensive review of the health care literature to 
develop a comprehensive bibliography of health care 
CEAs, CBAs, and related publications. 

The psychotherapy case study (Background Paper 
#3) used a separate, additional advisory panel that 
performed functions similar to the ones the overall 
advisory panel performed for the full assessment. 
The psychotherapy panel met on August 7, 1979, in 
Washington, D.C., to review the study plan, suggest 
improvements, and evaluate the background work 
performed. This panel met again on November 15, 
1979, to review and critique the initial draft of the 
case study. 

The third meeting of the advisory panel for the full 
assessment took place on October 16, 1979. In gener- 
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al, the panel focused on reviewing the status of the 
various parts of the study. It was also asked to dis- 
cuss the early drafts of the methodology report 
(Background Paper #1) and of the use of CEA in 
medicare reimbursement coverage. 

During 1979, two additional subprojects were initi- 
ated: a survey of analysts who had performed CEAs 
and CBAs of health care technologies (app. C), anda 
paper examining the ethical implications of CEA and 
CBA techniques (app. D). The survey of analysts fo- 
cused on the resource “costs” used to perform actual 
CEAs or CBAs. The analysts were also asked to esti- 
mate the expected or potential costs of an “ideal” 
study team doing CEA/CBA analyses of health care 
technology on a routine and continuing basis. The 
second subproject was a paper by the Hastings Insti- 
tute on the ethical considerations of conducting and 
using efficiency-based analyses such as CEA and 
CBA in the health care system. 

On January 18, 1980, the authors of the case stud- 
ies assembled in Washington, D.C., to review the ap- 
plicability of CEA/CBA to health care decisionmak- 
ing and to discuss the methodological or data prob- 
lems they faced in trying to apply CEA/CBA to their 
case study areas. Case study authors also discussed 
the policy issues involved in the use of CEA/CBA in 
health care decisionmaking. 

The final meeting of the advisory panel was held 
on March 28, 1980. At this meeting, the panel re- 


viewed drafts of the summary report, including the 
policy options for congressional consideration. Using 
the comments generated at this meeting, OTA staff 
revised the assessment report drafts. 

The results of this assessment are being issued in 
six volumes (described in app. F). A two- or three- 
tiered review process was used for each of the vol- 
umes and for each individual case study. The initial 
drafts were reviewed first by OTA staff and advisory 
panel members. In certain instances, outside review- 
ers were also asked for comments. After the authors 
completed their revisions based on the reviewers’ 
suggestions and comments, the drafts were sent out 
for a second round of review by a much broader 
range of experts in a diversity of settings: Federal 
agencies, State or local offices, private and nonprofit 
organizations, academic institutions, practicing 
health professionals, consumer groups, and other se- 
lected individuals. Altogether, more than 400 indi- 
viduals or organizations were asked to comment on 
drafts of case studies and other volumes of this as- 
sessment in the second round of review. The final re- 
port, the volume containing congressional options, 
was reviewed by more than 100. After appropriate 
revisions based on the comments received had been 
made, drafts were prepared for a final review by the 
assessment’s advisory panel, by the Health Program 
Advisory Committee, and by other individual re- 
viewers. 
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Introduction 


This appendix reports on the results of an OTA 
survey of the use of cost-effectiveness and cost-bene- 
fit studies by the major Federal health agencies, State 
and local governments, and nongovernmental orga- 
nizations. For the most part, OTA’s survey showed 
that CEA/CBA analyses are not frequently con- 
ducted or applied to health care decisionmaking. Fur- 
ther, when such analyses are used, they tend to be 
cost-benefit rather than cost-effectiveness analyses. 


Office of Assistant Secretary for 
Planning and Evaluation, DHHS 


In May 1966, John Gardner, Secretary of HEW 
(now DHHS), established five program analysis 
groups to conduct CBAs of several disease control 
programs. The objective of this effort was to provide 
a basis for comparing alternate programs and setting 
priorities for additional funding. Thus, an HEW offi- 
cial responsible for overseeing the analyses observed 
(266): 
HEW supports, or could support a number of cate- 
gorical disease control programs whose objectives are, 
or would be, to save lives or to prevent disability by 
controlling specific diseases. The studies were there- 
fore an attempt to answer the question: If additional 
money were to be allocated to disease control pro- 
grams, which programs would show the highest pay- 
off in terms of lives saved and disability prevented per 
dollar spent? 
The effort was originally undertaken in response to 
a request by the Bureau of the Budget for thorough 
analysis of the costs, benefits, and objectives of ex- 
isting and projected programs. That request reflected 
an interest in attempting to rationalize Government 
allocation decisions by building on the planning-pro- 
graming-budgeting system adopted by the Defense 
Department under Secretary McNamara (637). 
Ultimately, the five HEW program analysis groups 
produced several cost-benefit studies: 
1. Disease Control Programs: Arthritis, 1966. 
2. Disease Control Programs: Cancer, 1966. 
3. Disease Control Programs: Delivery of Health 
Services for the Poor, 1967. 

4. Disease Control Programs: Kidney Disease: 
Program Analysis, 1967. 

5. Program Analysis: Maternal and Child Health 
Care Programs, 1966. 

6. Application of Benefit-Cost Analysis to Motor 
Vehicle Accidents, 1966. 


7. Human Investment Programs: Vocational Re- 
habilitation, 1967.1 

In general, the impact of these studies appears to 
have been very limited by existing political and 
bureaucratic considerations, methodological short- 
comings in the analyses, and unrealistic expectations 
of the impact such evaluation studies could have on 
decisionmaking (637). 

One of the analyses, however, did have a major 
impact. The analysis of maternal and child health’ 
programs that examined the cost savings produced 
by federally sponsored periodic screening for low- 
income children played a major role in congressional 
passage of the Early and Periodic Screening, Diagno- 
sis, and Treatment (EPSDT) Program (H.R. 57100, 
1967) (637).2 The Johnson administration's interest in 
enacting a child health program created a receptive 
audience for the ideas presented in the analysis. Diffi- 
culties in the implementation of EPSDT have arisen 
over the years. One of the major obstacles has prov- 
en to be the high costs that States incur when they 
participate in the program (213). This problem was 
not focused on in the original examination of societal 
costs and benefits. 


National Institutes of Health 


NIH is the Government's principal biomedical re- 
search agency. Its 18 major organizational compo- 
nents support research on the causes, diagnoses, and 
treatment of diseases. Although a few institutes have 
engaged in economic analyses and cost-benefit 
studies, such efforts have been relatively atypical. 
The institutes have traditionally focused almost ex- 
clusively on gathering new knowledge as it relates to 
the disease process, having added an emphasis on 
assessing the safety and efficacy of new and existing 
technologies only recently. In addition, NIH has a 
very limited capacity to conduct economic analyses. 

These factors, among others, have combined to 
discourage NIH from conducting CEA or CBA stud- 
ies, as noted in an NIH staff memorandum (481): 


‘For a description of this effort, see R. N. Grosse and W. Smith, “Cost-Ef- 
fectiveness and Cost-Benefit Analysis for Public Health Programs,” Public 
Health Reports, 1968 (268). 

2For description of this analysis and the role played in the passage of 
EPSDT, see J. Wholey and G. Silver, Maternal and Child Health Care Pro- 
gram, 1966 (638); A. Levin, “Cost Effectiveness in Maternal and Child 
Health: Implications for Program Planning and Evaluation,” N. Eng. J. 
Med., 1968 (356); A. M. Foltz, “The Development of Ambiguous Federal 
Policy: Early and Periodic Screening, Diagnosis and Treatment (EPSDT)” 
(213), Milbank Memorial Fund Quarterly, 1975; and R. N. Grosse, ‘‘Cost 
Benefit Analysis of Health Services,” 1972 (266). 
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The Institutes of the NIH lack the staff, funds, and 
in-depth expertise to undertake large-scale economic 
analyses. Furthermore, safety and efficacy are still the 
primary concerns in the areas of disease research in 
which NIH is involved, the state-of-the-art in many of 
these areas is not yet sufficiently advanced to make 
cost-benefit analyses feasible. However, the NIH 
shares with agencies responsible for health care deliv- 
ery and regulation the mandate to evaluate new medi- 
cal procedures and devices, and where appropriate the 
Institutes do conduct small-scale cost-benefit /cost-ef- 
fectiveness analyses as components of more compre- 
hensive technology assessment studies. Extensive ef- 
forts at cost-benefit analyses would be more appropri- 
ately undertaken by other federal agencies within 
DHEW .. . which relate more closely with the health 
care [delivery] sector. 

Economic and cost-benefit studies that have been 
conducted by NIH have generally resulted from the 
interest of individual staff people (480). By far the 
most common types of economic analyses produced 
by NIH have been various cost-of-illness studies. 
These have been used in both planning and budget 
justifications. NIH cost-of-illness studies include: 

1. National Institutes of Health, Office of the Di- 
rector, “Trends of Economic Costs of Illness,” 
1979. 

2. National Cancer Institute, “Measurement of 
the Cost of Cancer Care,” 1978. 

3. P. A. Hoffstein, K. K. Krueger, and R. J. Wine- 
man, “Dialysis Costs: Results of a Diverse 
Sample Study,” Kidney International, 9:286, 
1976. [This is a summary of a study conducted 
for the National Institute of Arthritis, Metabo- 
lism, and Digestive Diseases by the Nephrology 
Cost Group. ] 

4. National Institute of Arthritis, Metabolism, 
and Digestive Diseases, “The Cost and Preva- 
lence of Complications of Dialysis in Five Di- 
alysis Centers,” 1978. 

5. E. J. Sondik, “Heart Disease, Magnitude of the 
Problem,” report prepared for the National 
Heart, Lung, and Blood Institute Task Force on 
Heart Disease in Childhood, July 3, 1979. 

6. National Institute of Neurological and Com- 
municative Disorders and Stroke, “Survey of 
the Incidence, Prevalence, and Cost of Inter- 
cranial Neoplasm,” 1979. 

The last study is part of a current effort by the Na- 
tional Institute of Neurological and Communicative 
Disorders and Stroke to link research priority setting 
more closely to identification of areas of greatest 
societal need and projects with the greatest potential 
for investment return. Thus, it is perceived as con- 
tributing to an effort to produce more “cost-effec- 
tive” research (480). Similarly, the cost-of-dialysis 
studies of the National Institute of Arthritis, Metabo- 


lism, and Digestive Diseases are perceived as having 
a cost-effectiveness element because they underline 
the different costs and complications of various 
modes of dialysis. 

Each institute of NIH seeks to reduce the morbidity 
and mortality associated with the disease under its 
mandate and considers research directions that are 
intended to reduce the human and economic toll of 
the diseases. Cost considerations have occasionally 
been considered in NIH consensus development con- 
ferences. Although their primary focus is on “clini- 
cally relevant research and the readiness of certain 
important findings to be put to use in health or reg- 
ulation,” these conferences may also consider social, 
ethical, legal, and cost implications. In general, how- 
ever, they consider these implications only as neces- 
sary extensions of their scientific and technical eval- 
uations of specific technologies. The principal re- 
sponsibility for examining these “nontechnical” 
issues rests with the National Center for Health Care 
Technology (NCHCT), which is discussed in a sepa- 
rate section of this appendix. The consensus devel- 
opment conferences sponsored by NIH serve only as 
the forum for the issues’ presentation. Consensus de- 
velopment conferences to date have addressed a wide 
range of subjects, including mammography, dental 
implants, and electronic fetal monitoring. 

In addition, three NIH institutes have produced 
more explicit cost-benefit work. One, the National 
Heart, Lung, and Blood Institute (NHLBI) has con- 
ducted several cost-benefit studies. NHLBI’s first ex- 
perience with CBA was in a 1973 assessment of heart 
transplant surgery and artificial heart development 
and use. As part of its long-term program of examin- 
ing mechanical circulatory technology, NHLBI issued 
a report on the medical, ethical, legal, financial, and 
social implications of artificial heart implantation 
(442). This assessment did not include a formal CBA, 
but did incorporate the concept by providing a de- 
tailed listing of the costs, as well as the medical and 
social benefits, of the artificial heart. The report's ex- 
amination of the cost of the device probably proved 
less important than its discussion of the formidable 
engineering obstacles to development, however, 
since it was these obstacles that played a major role 
in discouraging greater investment of resources by 
the institute in the totally implantable artificial heart 
program (514). 

In 1972, NHLBI’s mandate was broadened to in- 
clude prevention, education, and control responsibil- 
ities. Subsequently, the National High Blood Pres- 
sure Education Branch and the National High Blood 
Pressure Demonstration Program were created to 
work with Government and private agencies in an ef- 
fort to increase an awareness of the dangers of hyper- 
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tension and to encourage the development of effec- 
tive treatment. Both of these programs have utilized 
information produced by the cost-benefit studies to 
help convince State and local health agencies and pri- 
vate organizations of the value of instituting hyper- 
tension control programs (611,614). 

In 1975, NHLBI’s National High Blood Pressure 
Education Program conducted a CBA of a model hy- 
pertension control program in order to produce a 
model for analyzing the costs and benefits of a na- 
tional hypertension treatment program. This study 
was recently updated (441) through the incorpora- 
tion of new data and the development of a computer 
program which tests several of the major assump- 
tions made in the 1975 report (577). NHLBI has used 
information from this and the earlier analysis to en- 
courage effective treatment programs. 

Further, the National Institute of Dental Research 
(NIDR) has produced two cost-benefit studies. 
NIDR’s National Caries Program was established by 
Congress in 1971 to sponsor R&D activities directed 
toward ultimately reducing the incidence of dental 
disease in the American public. In carrying out this 
mission, program officials have used CBAs to assist 
them in their long-range planning of R&D investment 
decisions (83,522). In 1974, NIDR contracted for the 
development of a computer-based model for predict- 
ing the long-term “net social value” of the use of pre- 
ventive procedures (462). The results of the study 
were expressed in terms of the dollar savings pro- 
duced by dental care treatment on both a nationwide 
and regional basis. In 1978, an internal NIDR staff 
report assessed the costs and benefits of a specific 
caries prevention treatment, pit and fissure sealants 
(72). 

Both of these NIDR cost-benefit studies contrib- 
uted to a decision to halt investment in occlusal seal- 
ant clinical trials until a sealant and a system of deliv- 
ery could be developed that would significantly re- 
duce the clinical time required in sealant application 
(523). Examination of the “net social value” model 
also contributed to program decisions to increase 
funding of efforts aimed at reducing the high carcino- 
genicity of American diets and projects directed 
toward development of fluoride-based regimens for 
use in nonfluoride areas (523). 

Another study employing the cost-benefit concept 
has been funded by the National Cancer Institute 
(NCI). NCI originally contracted with the Blue 
Cross/Blue Shield Associations to develop a model 
prepaid health service benefit package for a cancer- 
screening program (435,436). Although originally the 
contents of the benefit package were to be deter- 
mined through the consensus of a panel of experts, 
the screening program that eventually was designed 


was largely determined by the use of cost-effective- 
ness methods (168). 

Cost-benefit considerations are often present in the 
various NIH institutes’ clinical trial deliberations. 
Although the evaluation of benefits is largely qual- 
itative, judgments about whether to undertake a clin- 
ical trial in order to test a given therapy often broad- 
ly consider costs and benefits. Perhaps the most sys- 
tematic and formal example of this process is pro- 
vided by NCI’s method of selecting new drugs for 
clinical trials. Within NCI, a large staff committee 
called the “Decision Network Group” engages in a 
formal evaluation of costs and potential benefits. A 
staff member describing the process observed (542): 

In making the selection of specific drugs to develop 

toward clinical trial, we consider the relative biologi- 
cal activity of the drug in comparison with the cost of 
development, including large-scale production. Thus, 
if the drug represents a new type of chemical class 
which is highly active and relatively inexpensive, we, 
of course, have no difficulty in reaching a decision. If, 
however, the new drug is of relatively marginal bene- 
fit in the experimental systems we utilize, and, in addi- 
tion, it is estimated to cost a great deal of money to 
produce and develop, we very seriously consider the 
advisability of proceeding. 

Another example of this type of cost consideration 

is a new drug related to one previously developed. If 

the original drug was very expensive and the new ana- 

log is considerably cheaper and of equivalent activity, 
we would very likely decide to switch to the new drug. 

Cost has a major role in the development and de- 

sign of the approaches we utilize for screening new 
potential antitumor drugs in experimental systems. 
Thus, in developing the panel of experimental systems 
currently utilized, we realized that it was impossible 
financially to carry out the testing in what might be 
considered the ideal scientific manner, namely, to test 
all materials in the complete battery of experimental 
systems. Since that approach was not financially fea- 
sible, we developed a pre-screening approach, so that 
the large number of compounds being tested would be 
evaluated in a relatively sensitive system first, and 
then further evaluated in the panel of experimental 
tumors based primarily on activity in that pre-screen. 

Another example of cost analysis involves the large 

animal preclinical toxicology studies carried out on a 

drug being developed for clinical trial. Our standard 

protocol for studies requires about 9-10 months and 
costs on the order of $120,000. We experimented with 
one very high priority drug to determine whether 
these experiments could be carried out in a shorter 
period of time. We found, indeed, that this could be 
done, but unfortunately, it required approximately 
twice as much money to carry out those experiments, 
since more tests had to be done simultaneously with 
some wasted effort in order to save time. Thus, it was 
decided that we would not routinely evaluate drugs in 
that manner since it was so expensive, but reserve 
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such alternate procedures for those very rare com- 
pounds that are considered of such high priority that 
they must be moved as rapidly as possible regardless 
of the cost... 

In summary, although we cannot carry out what 
one might like to see as completely quantitative cost 
benefit analyses, cost considerations have always 
been and remain a vital part of our everyday life in 
making scientific decisions and in attempting to devel- 
op new and better modes of therapy in the shortest 
amount of time with the funds available to use. 
Although other NIH institutes utilize a less formal 

selection process than NCI, cost-benefit considera- 
tions often enter into their decisions concerning the 
need for clinical trials. A staff member of NHLBI, for 
example, cited three instances in which cost consid- 
erations played a part in such decisions? (577). 

A number of the Institute's clinical trials were un- 
dertaken for reasons very much concerned with the 
potential benefits to be accrued from a careful testing 
of a given therapy or preventive regimen. In particu- 
lar, the coronary artery surgery study (CASS) is to de- 
termine the efficacy of surgical vs. medical interven- 
tion for coronary artery disease . . . Coronary artery 
surgery (by-pass surgery) is an extremely expensive 
operation (between $10,000 and $20,000 per case) that 
may be able to be treated as efficaciously and at con- 
siderably less cost through non-surgical means. In ar- 
riving at the decision to undertake CASS, the Institute 
weighed the potential benefits of the information to be 
derived against the considerable cost of the study. In 
similar fashion, the Institute considered a trial related 
to mild hypertension treatment, but decided that the 
Institute’s current portfolio of clinical trials related to 
hypertension would provide much the same informa- 
tion as a new trial. It was calculated that such a trial 
was not currently warranted. The Extracorporeal 
Membrane Oxygenator (ECMO) and Intermittent 
Positive Pressure Breathing (IPPB) trials are two addi- 
tional NHLBI supported trials concerned with deter- 
mining the effectiveness of costly, unvalidated ther- 
apies. 


Food and Drug Administration 


FDA is responsible for monitoring the safety of 
foods and cosmetics and evaluating the safety and ef- 
ficacy of drugs and medical devices, food, feed, and 
color additives. With the exception of several studies 
performed by the Bureau of Radiological Health 
(BRH), FDA's experience with CEA is limited to eight 
analyses conducted in response to regulatory initia- 
tives.* 

3For a discussion of NHLBI clinical trial planning and decisionmaking, 
see R. I. Levy and E. J. Sondik, “Decision-Making in Planning Large-Scale 
Comparative Studies,” and R. I. Levy and E. J. Sondik, “Initiating Large- 
Scale Clinical Trials,” 1978. 

‘For other regulatory agencies’ experience with these requirements, see T. 


B. Clark, “Its Still No Bureaucratic Revolution, But Regulatory Reform Has 
Foothold,” National Journal, Sept. 29, 1979 (97). 


On March 23, 1978, Executive Order 12044 man- 
dated that Government agencies examine the costs 
and benefits of major proposed regulations (i.e., reg- 
ulations having an annual impact on the economy of 
“$100 million or more or [causing] a major increase 
in costs or prices for individual industries, levels of 
government or geographic regions”). This mandate 
was built upon an earlier executive order issued by 
President Ford which required an “inflation impact” 
analysis of major proposed regulations in order to 
assure that the private sector not be burdened by un- 
justified costs.° 

Under these mandates, FDA has conducted eco- 
nomic analyses of eight proposed initiatives (218): 

1. Food and Drug Administration, Office of Plan- 
ning and Evaluation, Office of the Commis- 
sioner, “Inflation Impact Statement of Pro- 
posed Rulemaking: Certain Fluorocarbons and 
Chlorofluorocarbons in Food, Food Additives, 
Drug, Animal Food, Animal Drug, Cosmetic 
and Medical Device Products as Propellants in 
Self-Pressurized Containers, Prohibition on 
Use,” May 1977. 

2. Food and Drug Administration, Office of Plan- 
ning and Evaluation, Office of the Commis- 
sioner, “Inflation Impact Statement of Pro- 
posed Rulemaking: Nitrofuran (S-Nitro) Com- 
pounds,” May 1976. 

3. Food and Drug Administration, Office of Plan- 
ning and Evaluation, Office of the Commis- 
sioner, “Inflation Impact Statement of Pro- 
posed Rulemaking: Diethylstilbestrol,” January 
1976. 

4. F. H. Dworkin, Economics Staff, Office of 
Planning and Evaluation, Food and Drug Ad- 
ministration, “An Analysis of Economic Im- 
pact of the Drug Regulation Reform Act of 
1978,” November 1978. 

5. F. H. Dworkin, Economics Staff, Office of 
Planning and Evaluation, Food and Drug Ad- 
ministration, “Chemical Compound in Food- 
Producing Animals: Criteria and Procedures 
for Evaluating Assays for Carcinogenic Resi- 
dues,” March 1979. 

6. Food and Drug Administration, Office of Plan- 
ning and Evaluation, Office of the Commis- 
sioner, “Inflation Impact Statement of Pro- 
posed Rulemaking: Saccharin and Its Salts in 
Foods, Drugs, Animal Feeds, Animal Drugs, 
Cosmetics, and as a Food Additive, Prohibition 
of Uses,” April 1977. 

7. P. F. Lewis, Environmental and Economic Im- 
pact Staff, Bureau of Foods, Food and Drug 


®This earlier executive order was Executive Order 11821, superseded by 
Executive Order 11949, 1977. 


Administration, “Regulatory Analysis for Final 
Regulation for Reduction of Temporary Toler- 
ances for Polychlorinated Biphenyls in Food,” 
June 27, 1977. 

8. F. H. Dworkin, et al., Economics Staff, Office 
of Planning and Evaluation, Food and Drug 
Administration, “Draft Regulatory Analysis: 
Patient Labeling Requirements for Prescription 
Drug Products,” June 1979. 

In most of these economic analyses, however, FDA 
was only able to list benefits. In general, it was not 
able to quantify them owing to a lack of adequate 
data, methodological problems, and difficulty in pro- 
jecting behavior precipitated by proposed manufac- 
turing controls (43,646). Further, when an analysis 
involves a carcinogenic food additive, as it did in 
three instances,® FDA is legally forbidden to weigh 
costs and benefits in making a decision. According to 
the “Delaney clause” of the Food, Drug, and Cosmet- 
ic Act of 1938,’ if a food additive is found to be car- 
cinogenic, FDA must ban it.* Nevertheless, FDA 
seems to have found economic analyses generally 
useful, and the consideration of specific benefits 
probably has contributed to the modification of some 
provisions of several proposed regulations (43).° 1° 

One of FDA's component bureaus, BRH, has used 
economic analyses and CBAs more extensively than 
other component bureaus. BRH is responsible for 
protecting the public from unnecessary exposure to 
radiation and ensuring that radiation is used safely 
and efficaciously. In 1977, BRH conducted an “Eco- 
nomic Impact Assessment of the Proposed Perform- 
ance Standard for Sunlamp Products,” January 1977. 
Because of the limited projected impact of this regula- 
tion, BRH assessed only costs. 

In April of 1977, however, BRH used the cost- 
benefit concept more specifically, to justify budget 
requests for expanding the Bureau’s effort to reduce 
the unnecessary use of X-rays. Thus, the fiscal year 
1979 Preliminary Budget Justification stated (217): 

The proposed national X-ray system will cost an ad- 
ditional $7.7 million contract dollars and 181 posi- 

tions. This gives a benefit/cost ratio ranging from 32 

to 1 to 70 to 1 for the 20-50 percent possible reductions 

in the genetically significant dose alone. 


‘The inflation impact statements for nitrofuran, diethylstilbestrol, and 
saccharin. 

7See Office of Technology Assessment, U.S. Congress, Cancer Testing 
Technology and Saccharin, October 1977 (466). 

*This legislation takes precedence over Executive Order 12044. 

°The cost of releasing safety and efficacy data on new drugs, revealed by 
the analysis of the 1978 Drug Regulatory Reform Act, for example, contrib- 
uted to the dropping of the draft requirement that such information be re- 
leased. 

1 Another case in which economic analysis contributed to a modification 
of policy is FDA's action on antibiotics and feed: “Combined Economic Im- 
pact Assessment of Series of Proposals Regarding Penicillin/Tetracycline- 
Containing Premixes,” undated. 
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This cost-benefit argument was the “driving force” 
(341) behind the development of a cost containment 
plan: “A Proposed FDA Program To Reduce Unnec- 
essary Patient Exposure From Diagnostic X-Rays: 
Cost Containment Considerations” August 1978. In 
this plan, CBA was used to demonstrate the cost sav- 
ings produced by reducing unnecessary X-rays. 

Another CBA performed by BRH was “The Diag- 
nostic X-Ray Equipment Performance Standard and 
the Policy on Assembly and Reassembly,”” November 
1978. This study represented a refinement and elabo- 
ration of an earlier attempt (216) to examine the costs 
and benefits of the X-ray equipment standard in an 
environmental assessment report in 1974. The results 
of this updated study contributed to FDA's decision 
to amend its policy on the assembly and reassembly 
of diagnostic X-ray equipment and revoke two reg- 
ulatory provisions which were shown not to be cost 
beneficial (215,340). 


Center for Disease Control 


CDC is responsible for monitoring, controlling, 
and reducing the incidence of preventable diseases 
and conditions. Over the years, CDC has conducted 
a number of assessments of the costs of illnesses and 
treatment methods, as well as cost-benefit studies. In 
1966, CDC involvement in the HEW program anal- 
yses of the costs and benefits of tuberculosis and 
syphilis control promoted staff interest in examining 
the economic impact of disease.1! Since then, CDC 
has often used economic analysis to supplement the 
Center’s traditional public health perspective by 
allowing the costs of a disease to be considered along 
with morbidity and mortality statistics. Since much 
of CDC’s work involves providing information to 
State and local public health officials who must be 
concerned with budgets, economic analysis has 
proved particularly helpful in supporting the value of 
suggested disease prevention control programs (32, 
601). A CDC official noted: 

[These analyses] enable the cost of disease control 
efforts and investigations to be placed in perspective. 
For vector-borne diseases (dengue, equine encephali- 
tis), such analyses make it easier to justify costly 
spraying and clean-up efforts. 

The economic impact studies conducted by CDC 
staff include: 

1. D.J. Sencer and N. W. Axnick, ‘Utilization of 
Cost Benefit Analysis in Planning Prevention 
Programs,” Acta Medica Scandinavia, Supple- 
ment 576, 1975. [This study reports on the 


"See the section in this appendix on the Office of the Assistant Secretary 
for Planning and Evaluation, DHHS. Also, for a discussion of CDC’s anal- 
yses of syphilis and tuberculosis control, see R. N. Grosse, “Cost-Benefit 
Analysis of Health Services,” Ann. Am. Ac., 1972 (266). 


150 © The Implications of Cost-Effectiveness Analysis of Medical Technology 


10. 


il. 


12. 


13: 


costs of hepatitis, hospital infections, and the 
1964-65 rubella epidemic. | 

D. D. Tolsma and J. A. Bryan, “The Economic 
Impact of Viral Hepatitis in the United States,” 
Public Health Reports 91:349, 1979. 

S. D. Von Allmen, R. H. Lopez-Correa, J. P. 
Woodall, D. M. Morens, J. Chiniboga, and A. 
Casta-Valez, “Epidemic Dengue Fever in Puer- 
to Rico, 1977: A Cost Analysis,” American 
Journal of Tropical Medicine, November 1979. 
E. L. Baker, Jr., W. Peterson, S. D. Von AIll- 
men, and J. Fleming, ‘Economic Impact of a 
Community-Wide Waterborne Outbreak of 
Gastrointestinal Illness.” 

M. L. Cohen, R. E. Fontaine, R. A. Hollard, S. 
D. Von Allmen, T. M. Vernon, and E. J. Gan- 
garosa, “An Assessment of Patient-Related 
Costs in an Outbreak of Salmonellosis.” 

D. D. Tolsma and J. D. Millar, “Costs and 
Accomplishments of the United States Nation- 
al Influenza Immunization Program, 1976,” 
Eighth International Scientific Meeting of the 
International Epidemiologic Association Sym- 
posium on Epidemiologic Aspects of Cost-Ef- 
fectiveness Studies, San Juan, Puerto Rico, 
Sept. 19, 1977. 


. R.C. Rendtorff, J. C. Curran, R. W. Chandler, 


W. L. Wiser, and H. Robinson, “Economic 
Consequences of Gonorrhea in Women: Ex- 
periences From an Urban Hospital,” Journal of 
the American Venereal Disease Association, 
1(1), September 1974. 


. S.D. Von Allmen, W. Cates, Jr., K. F. Schulz, 


D. A. Grimes, and C. W. Tyler, Jr., “Cost of 
Treating Abortion-Related Complications,” 
Family Planning Perspectives, 9(6), Novem- 
ber/December 1977. 


. J. L. Bradley, “The Cost of Diphtheria Epidem- 


ic in San Antonio, Texas, 1970-71,” paper pre- 
sented at the American Public Health Associa- 
tion Meeting, San Francisco, Nov. 7, 1973. 

P. M. Schwab, “Economic Cost of St. Louis En- 
cephalitis Epidemic in Dallas, Texas, 1966,” 
Public Health Reports, 83(10):860, October 
1968. 

B. S. Levy, “The Economic Impact of a Food- 
Borne Salmonellosis Study,” Journal of Ameri- 
can Medical Association, 230:1281, Dec. 2, 
1974. 

N. W. Axnick, “Cost of 1968-69 Influenza Epi- 
demic,” paper presented to the Surgeon Gener- 
al’s Advisory Committee on Immunization 
Practice, April 1970. 

S. D. Von Allmen, R. H. Lopez-Correa, J. P. 
Woodall, D. M. Morens, J. Chiniboge, and A. 


Casta-Valez, “Epidemic Dengue Fever in Puer- 
to Rico, 1977: A Cost Analysis,” American 
Journal of Tropical Medicine, November 1979. 

14. E. L. Baker, Jr., W. Peterson, S. D. Von All- 
men, and J. Fleming, “Economic Impact of a 
Community-Wide Waterborne Outbreak of 
Gastrointestinal Illness.” 

15. M.L. Cohen, R. E. Fontaine, R. A. Pullard, S. 
D. Von Allmen, T. M. Vernon, and E. J. Gan- 
garosa, “An Assessment of Patient-Related 
Costs in an Outbreak of Salmonellosis.” 

16. N. W. Axnick and J. M. Lane, “Costs Associ- 
ated With the Protection of the United States 
Against Smallpox,” World Health Organiza- 
tion Report, March 1973. 

17. P. M. Layde, S. D. Von Allmen, and G. P. 
Oakley, “Antenatal Screening for Downs Syn- 
drome: An Economic Viewpoint,” September 
1979. 

18. J. J. Witte and N. W. Axnick, “The Benefits 
From 10 Years of Measles Immunization in the 
United States,” Public Health Reports, 90:3, 
May-June 1975. 


In recent years, CDC also has conducted a number 
of cost-benefit studies. Several CDC officials noted 
that CBA contributes an additional dimension to 
more traditional evaluations of morbidity and mor- 
tality, and where adequate data exist, it has become 
logical to consider this information an evaluation. In 
some cases, CBA seems to provide a particularly 
meaningful perspective, because humanitarian con- 
cerns can be weighed against dollar costs and the 
value of prevention can be made more tangible by 
describing benefits in terms of averted costs. As a 
consequence, CBA has been used by CDC staff to 
plan and evaluate prevention and control programs 
and justify investment in such efforts (32,325). 


Cost-benefit assessments conducted for the CDC 
staff include: 


1. P. M. Layde, S. D. Von Allmen, and G. P. 
Oakley, “Maternal Serum Alpha-Fetoprotein 
Screening: A Cost Benefit Analysis,” American 
Journal of Public Health, 69(6):566, June 1979. 

2. P. M. Layde, S. D. Von Allmen, and G. P. 
Oakley, “Congenital Hypothyroidism Control 
Programs: A Cost-Benefit Analysis,” Journal of 
the American Medical Association, 24(21): 
2290, May 1979. 

3. P. V. Strange and A. T. Sumner, “Predicting 
Treatment Costs and Life Expectancy for End- 
Stage Renal Disease,” N. Eng. J. Med., 298: 
372, Feb. 16, 1978. 

4. D. J. Sencer and N. W. Axnick, “Cost Benefit 
Analyses,” Proceedings of the 145th Sympo- 
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sium on Vaccination Against Communicable 
Diseases, Monaco, March 1973. 

5. J. P. Koplan, S. Schoenbaum, M. Weinstein, 
and D. Fraser, “Pertussis Vaccine: An Analysis 
of Benefits, Risks, and Costs,” N. Eng. J. Med., 
Oct. 25, 1979. 

6. J. K. Paperfuss and B. C. Ejeldsled, “Cost-Ben- 
efit Study of Selected Interventions in Control 
and Prevention of Tuberculosis in the State of 
Michigan,” study prepared by the Michigan 
Division of Health for CDC, December 1969. 


Alcohol, Drug Abuse, and 
Mental Health Administration 


ADAMHA has major responsibility for the pre- 
vention and treatment of mental illness, alcohol 
abuse, and drug abuse. One of ADAMHA’s three 
component bureaus, the National Institute on Drug 
Abuse (NIDA), has produced several cost-of-illness 
and cost-benefit studies. They include: 

1. Arthur D. Little, “Social Cost of Drug Abuse,” 

1976. 

2. Macro Systems, Inc., “Unit and Episodic Costs 
of Drug Abuse Treatment: A_ Feasibility 
Study,” May 1978. 

3. Center for Social Research on Drug Abuse, 
“The Assessment of Social Cost Savings,” 
March 1978. 

4. Research Triangle Institute, “Management Ef- 
fectiveness Measures for NIDA Drug Abuse 
Treatment Programs: Costs to Society,” vol. II, 
1977. 

These studies have been used in both planning and 
budget justification. The cost information yielded by 
the analyses has provided basic data for CEA/CBA 
of drug abuse treatment. 

In 1972, the U.S. Bureau of Narcotics and Danger- 
ous Drugs conducted a study of the costs and benefits 
of alternative approaches to addiction control (386). 
A study funded by NIDA in 1975 extended this idea 
by developing four primary measures of the effec- 
tiveness of drug treatment and comparing the cost- 
effectiveness of five treatment modalities: methadone 
maintenance, therapeutic community, inpatient de- 
toxification, outpatient detoxification, and drug-free 
treatment (535a). 

More recent studies have focused on the develop- 
ment of a more sophisticated cost-effectiveness and 
cost-benefit ratios for drug abuse treatment programs 
(504) and examination of the applicability of CEA to 
primary drug abuse prevention programs (298,505). 
These recent NIDA analyses are acknowledged to 
have serious methodological limitations because of 
inadequate data about such factors as the total num- 


ber of drug abusers, the relapse rate of treated 
abusers, the relationship between drug abuse and un- 
employment and crime, and the relationship between 
patient outcome and treatment modality (535). 

Nevertheless, the cost-benefit/cost-effectiveness 
studies are credited with providing valuable informa- 
tion for program evaluation and management and 
NIDA budget justification (360,520,534,535a). The 
studies have also confirmed the worth of allocating 
more funds to the less traditional and expensive treat- 
ment approaches (i.e., of a funding formula which 
favors outpatient methadone maintenance, outpa- 
tient drug detoxification, and outpatient drug-free 
programs rather than inpatient detoxification and 
residential community approaches). In addition, they 
have been useful in confirming the benefits produced 
by the Federal investment in drug abuse treatment. 
As an official of NIDA observed (535a): 

. .. Based on an empirical analysis of data, we 
have established average gains per patients year on 
performance criteria and can translate these gains into 
cost savings benefits to society. In turn, we can com- 
pare them to what society has invested in treatment. 
The conclusion is that it pays to invest in treatment. 
Unfortunately, such analyses are still severely lim- 

ited by their inability to specify which treatment mo- 
dality tends to work best for a given type of client. In 
order to better address this question, NIDA is cur- 
rently attempting to collect more accurate data on 
drug abuse and rehabilitation patterns through the 
Drug Abuse Reporting Program (DARP) and the 
Treatment Outcome Prospective Study (TOPS). 

At the National Institute of Mental Health 
(NIMH), there are two potential applications of 
CEA/CBA studies: 1) in measurement of service pro- 
gram outcomes, such as the community mental 
health centers (CMHCs) that receive Federal spon- 
sorship, and 2) the measurement or evaluation of 
psychosocial interventions. It is important to keep 
these two quite different research levels separated. 
Depending on the research focus—program out- 
comes or psychosocial treatment outcomes—there 
will be differences in what is measured, the means of 
measurement, the purpose of measurement, the uses 
of the information, and the types of people involved 
in the research. 

The Psychotherapy and Behavioral Intervention 
Section of NIMH’s Division of Extramural Research 
Programs is responsible for funding research on the 
effectiveness, efficacy, safety, etc., of psychosocial 
treatment mechanisms for specific mental disorders. 
Although one or two of the NIMH-sponsored psy- 
chotherapy outcome studies have examined variables 
related to cost benefit or cost effectiveness (e.g., du- 
ration of treatment required), most such studies are 
not intended or designed to address cost variables. 
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Traditionally, psychotherapy research seeks only to 
elucidate the mechanisms of action and the efficacy 
of specific treatments for specific diagnostic catego- 
ries of mental disorders. The Psychotherapy and Be- 
havioral Intervention Section is currently attempting 
to encourage the use of a range of standard proce- 
dures for the measurement of change in psychother- 
apy in the hope of achieving greater comparability in 
the assessment of outcomes across various studies. 
Cost-effectiveness variables, though, are even more 
difficult to apprehend and often are not useful at this 
level of research. 

The Division of Biometry and Epidemiology at 
NIMH has conducted several studies of the services 
programs and episodes-of-illness costs of patients 
treated in federally sponsored CMHCs. Costs and 
cost-effectiveness criteria appear to be potentially rel- 
evant for use in evaluating program costs and exam- 
ining the outcomes of delivering mental health serv- 
ices at the community level. NIMH has funded stud- 
ies under contract which seek to advance the meth- 
odological framework of CBAs as applied to 
CMHCs’ programs (443). NIMH (or the Division) 
does not conduct such studies itself because of the 
limited development of the methods, differences in 
CMHC’ clientele, and the tendency for such assess- 
ments to be subject to misrepresentation and to be 
used to criticize the management of individual 
CMAGs 2 

NIMH has supported training courses in CBA 
through its staff .college, and through the contract 
mechanism, is supporting several State governments 
in the development of management systems that will 
provide unit cost data for mental health service sys- 
tems. A few States (Colorado, Oregon,!* and Wash- 
ington) have officials who are working with NIMH 
to develop a more sophisticated methodology for ex- 
amining CMHC program costs and outcomes. 


National Center for Health Care 
Technology 


(See ch. 9 for a discussion of NCHCT.) 


2For example, see D. E. Carter and F. L. Newman, A Client Oriented 
System of Mental Health Service Delivery and Program Management: A 
Workbook and Guide, 1976 (85); J. E. Sorensen and D. W. Phillips, “Cost 
Finding and Rate-Setting for Community Mental Health Centers,” 1972 
(578); and W. Hargeaves, et al., Resource Materials for Community Mental 
Health Program Evaluation. 

13See G. Brodsky and D. A. Bigelow, The Oregon Program Impact Moni- 
toring System, 1978 (70). 


National Center for Health Services 
Research 


NCHSR funds a number of CEA and CBA studies 
as part of its services research function (439). It is 
currently the major supporter of CEAs and CBAs in 
the Federal health care research system. Two of 
NCHSR’s research priority areas—health care costs 
and cost containment, and planning and regulation— 
specifically call for research using CEA studies to ex- 
amine the issues in these priority areas (438). 
The types of economic analyses NCHSR supports 
range from what could be considered traditional 
CEAs or CBAs to analyses that focus on either costs 
or outcomes of a given technology or program. It is 
difficult to characterize the range of economic studies 
NCHSR supports. The studies range from rigorous 
examinations of specific health care topics to broader 
studies of more complex health care issues. CEA or 
CBA may be the focus of a grant or may be incorpo- 
rated into larger, more global assessments. Never- 
theless, NCHSR is a significant source of support for 
these types of techniques. Some of the more recent 
CEA and CBAs that have received support from 
NCHSR are listed below (439): 
1. “Impact of Ophthalmic Technicians on Outpa- 
tient Care,” HS 03647. 

2. “Improving Clinical Decision Making in Arth- 
ritis,”” HS 03115. 

3. “Prediction of Cost Effective Stroke Rehabilita- 
tion,” HS 03693. 

4. “Controlled Trial of a Quality Assurance 
Mechanism,” HS 02485. 

5. “Evaluation of Breast Cancer Detection Strate- 
gies,” HS 03256. 

6. “Cost Effective Strategies in Ambulatory 
Care,” HS 02063. 

7. “A Guide to Investment Criteria for Critical 
Care Units,” HS 03569. 


National Center for Health Statistics 


NCHS is an agency within the Office of the Assist- 
ant Secretary for Health of DHHS. It is one of the 
principal health services research agencies of the Fed- 
eral Government. Health statistics activities, which 
eventually were formalized and combined by the cre- 
ation of NCHS, were authorized by Congress in 1946 
(vital statistics) and in 1956 (National Health Sur- 
vey). NCHS was formed in 1960 and has played a 
major role in the development of national health sta- 
tistics policy and programs. Under its current 
mandate—the Health Services Research, Health Sta- 
tistics, and Medical Libraries Act of 1974 (Public Law 
93-353)—NCHS is responsible for collecting and dis- 


seminating health data including information on the 
costs of illness, health care, and health financing. 

The importance of data collected by NCHS cannot 
be overemphasized. Information such as that con- 
cerning the incidence and prevalence rates of dis- 
eases, natural history of disease, medical care utiliza- 
tion, workloss, surgical rates, and premature mortal- 
ity is crucial to CEA/CBA. Although NCHS does not 
conduct full cost-effectiveness studies, it has con- 
ducted cost-of-illness studies, and these have been 
used in other agencies’ CEA/CBAs. Currently, 
NCHS staff are coordinating an interagency Public 
Health Service (PHS) Cost of Illness Committee 
which is examining the state-of-the-art of estimating 
costs of illness and disability. As noted by an NCHS 
official, it has often proved impossible to compare 
the results of cost-of-illness studies conducted by 
NCHS and other DHHS agencies (297): 

Frequently . . . the assumptions, methods and data 
employed vary to such an extent that estimates from 
two different studies differ markedly, and the costs of 
several illnesses cannot be compared. 

In an attempt to address this problem, the PHS 
Cost of Illness Committee will recommend a set of 
guidelines for future PHS studies. 


Health Resources Administration 


HRA is responsible for improving the national ca- 
pacity to develop and effectively use health re- 
sources. Its Office of Program Planning and Evalua- 
tion has sponsored conferences which produced two 
studies focusing on cost-effectiveness methodology: 

1. S. O. Schweitzer, ‘The Economics of the Early 
Diagnosis of Disease.” [This article presents a 
methodological framework for the evaluation 
of the cost-effectiveness of early diagnostic 
tests. | 

2. H. A. Foley, D. B. Fisherman, and F. Kalibat, 
“The Cost-Effectiveness of a Mental Health 
Center: An Experiment,” in Evaluation in 
Health Services Delivery, edited by R. Yaffe 
and D. Zalkind, proceedings of an Engineering 
Foundation Conference, South Berwick, 
Maine, Aug. 19-24, 1973. 

The Bureau of Health Manpower (BHM), one of 
HRA’s component agencies, has also produced one 
cost-effectiveness study: 

1. N. Doherty, “Study to Determine the Cost and 
Effectiveness of Different Practice Modes of 
Dental Care for Children,” February 1977. 

Another HRA agency, the Bureau of Health Plan- 
ning (BHP), has sponsored two cost-benefit studies: 

1. Arthur D. Little, “Introduction to Cost-Benefit 
Analysis Applied to New Health Technol- 
ogies,” December 1977. 
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2. Stanford Research Institute, ‘Decision Making 
Methodology for Health Care Services,” study 
prepared for the Region IX (San Francisco) Of- 
fice of DHEW, December 1979. 

Title V of the National Health Planning and Re- 
sources Development Act of 1974 (Public Law 
93-641) suggests that local health planning agencies 
(health systems agencies (HSAs)) should consider the 
costs and benefits of projects (sec. 1514(3)): 

In establishing the Annual Implementation Plan, 
the agency shall give priority to those objectives 
which will maximally improve the health of the resi- 
dents of the area, as determined on the basis of the re- 
lation of the cost of attaining such objectives to their 
benefits, and which are fitted to the special needs of 
the area. 

But most HSAs do not appear to use CBA.** The two 
guidebooks to cost-benefit methodology which BHP 
has funded constitute only a small segment of the 
technical assistance literature which is being distrib- 
uted to HSAs. 


Health Services Administration 


The Health Services Administration is responsible 
for Federal programs that provide health care serv- 
ices to specific populations. It has sponsored a num- 
ber of studies that deal with cost-benefit considera- 
tions: 

1. Abt Associates, Inc., “Analysis of Benefits and 
Costs of Ambulatory Care Services Offered by 
the Bureau of Community Health Services,” 
March 1975. 

2. Geomet, Inc., “Comparison of Cost-Per-Day in 
PHS Hospitals: An Approach to Service and 
Cost Comparisons,” September 1975. 

3. Health Services Administration/OPEL and 
Geomet, Inc. (A. Dobson and H. L. Heaton), 
“An Economic Analysis of HSA Program Po- 
tential, 1976-1981,” report No. OPEL 75-4, 
May 1975. 

4. Health Services Administration/OPEL and 
Geomet, Inc. (I. Enger and N. Pindus), “Cost 
and Utilization Evaluation Migrant Hospital 
Program,” report No. OPEL 77-1, January 
1977. 

5. Health Services Administration/OPEL and 
Geomet, Inc., “Improved Pregnancy Outcome 
Program Evaluation Manual,” report No. 
OPEL 77-2, March 1977. 

6. Health Services Administration/OPEL, 
“PSRO: An Evaluation of the Professional 
Standards Review Organization Program,” 
vol. 1-11, report No. OPEL 77-12, October 
LOTT 


14For further discussion of this subject, see ch. 6 of this report. 
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7. Transaction Systems, Inc., “Credentialling of 
Health Care Professionals: Its Costs and Some 
Alternatives for the Health Services Adminis- 
tration,” March 1975. 

8. General Research Corporation, “PSRO Pro- 
gram Cost Study,” April 1974. 

9. Geomet, Inc., “Cost and Utilization Evaluation 
of Migrant Ambulatory Care Entitlement Pro- 
grams,” March 1979. 

10. Geomet, Inc., “Comparative Cost and Finan- 
cial Analysis of Ambulatory Care Providers,” 
June 1976. 

11. Health Services Administration, “Migrant Am- 
bulatory Care Entitlement Programs - Cost and 
Utilization Evaluation of Rural Health Initia- 
tive,”” October 1978. 


Health Care Financing Administration 


HCFA is responsible for administering the medi- 
care, medicaid, Professional Standards Review Orga- 
nization (PSRO), and End-Stage Renal Disease Pro- 
grams. It has conducted several studies of the eco- 
nomic toll of illness and the cost of health care fi- 
nancing. These include: 

1. B. S. Cooper and D. P. Rice, ‘The Economic 

Costs of Illness Revisited,” Social Security Bul- 
letin, 39(2), February 1976. 

2. D. P. Rice, “Estimating the Cost of Illness,” 
Health Economics Series No. 6, Public Health 
Service, 1966. 

3. R. M. Gibson and C. R. Fisher, “National 
Health Expenditures, Fiscal Year 1977,” Social 
Security Bulletin, 41(7):3, 1978. 

In addition, HCFA is currently sponsoring an effort 
aimed at encouraging the integration of cost contain- 
ment information into medical school curricula: 

1. American Association of Medical Schools, “A 
Primer on Quality Assurance and Cost Con- 
tainment for Faculty and Students.” 

Also, HCFA has sponsored a study which indirectly 
employed the cost-effectiveness concept: 

1. Massachusetts Department of Public Welfare, 
“Cost Effectiveness Under Medicaid,” 1978. 

This study was designed to investigate whether medi- 
cal criteria could be utilized within a medicaid pro- 
gram to reduce inappropriate and ineffective medical 
care. Problems in implementation, however, led to 
HCFA’s termination of funding for the study. 

In addition, HCFA has conducted an assessment of 
the costs and the benefits of PSROs. Under Title XI of 
the Social Security Act, PSROs are responsible for 
reviewing the medical necessity, quality, and appro- 
priateness of federally financed health programs. In 
order to perform this function, PSROs must develop 


norms, criteria, and standards for the appropriate 
utilization and acceptable quality of health care serv- 
ices. A 1979 report by HCFA, elaborating on a previ- 
ous evaluation (288), concluded that the savings pro- 
duced by PSROs through reduced use of hospital 
services by medicare patients exceed the cost of 
PSRO review (285). Subsequently, this finding was 
used in PSRO program justification before Congress. 

Two other Government evaluations of PSROs, 
however, have challenged HCFA’s findings. One, a 
study by the Congressional Budget Office found that 
PSROs only slightly reduce medicare patients’ use of 
hospital services and concluded that the “PSRO pro- 
gram probably yields a net loss” (121). Similarly, a 
1979 General Accounting Office study stated that the 
data HEW reviewed in its 1977 and 1978 evaluations 
“are not based on appropriate hospital statistics” and 
that several estimates of cost savings attributable to 
PSROs were overstated (245). 


Office of Human Development Services 


OHDS of DHHS administers a wide range of pro- 
grams which are designed to aid children, youth, 
families, the aged, the handicapped, and native 
Americans. Over the last few years, since the first ap- 
plication of cost-benefit framework to vocational 
rehabilitation by Ronald Conley in 1965 (124), 
OHDS has conducted a number of CBAs of rehabili- 
tation services. OHDS, as well as many State voca- 
tional rehabilitation agencies, has used this type of 
analysis to justify increased Government spending on 
training programs for the disabled. 

The application of CBA to vocational training ap- 
pears to have been facilitated by the existence of both 
a traditional set of program goals (e.g., gainful em- 
ployment of the disabled) and a long-term informa- 
tion system about such factors as increased earnings 
and reduced costs of special medical or custodial care 
(479). Among the many federally and State-spon- 
sored studies of this type are the following: 


Federally Sponsored Studies 


1. Rehab Group, Inc., ‘Feasibility Study for the 
Evaluation of Methodologies for Cost Benefit 
Analysis of Restoration Services in Rehabilita- 
tion,” prepared for OHDS, October 1978. 

2. R. O. Washington, D. Y. Rowland, C. Hairston, 
“Quantification of Human Services Outcomes: A 
Manual for Applying Program Budgeting, Sys- 
tems Analysis and Cost-Benefit Analyses to Hu- 
man Service Programs,” prepared for Social and 
Rehabilitation Services, August 1974. 


3. L. Mars, “An Exploratory Cost-Benefit Analysis 
of Vocational Rehabilitation,” prepared for the 
Vocational Rehabilitation Administration, 1967. 

4. D.E. Carter and T. J. George, “A Manual for Re- 
porting and Analyzing the Costs and Benefits of 
Social Services,” prepared for Social and Rehabil- 
itation Services, November 1973. 

5. H. Emlet, et al., “Estimated Health Benefits and 
Cost of Post-Onset Care for Stroke,” prepared by 
Analytic Services, Inc., in cooperation with Johns 
Hopkins University, for Social and Rehabilitation 
Services, September 1973. [This is an assessment 
of the cost and benefits of poststroke care in the 
population of three States. ] 

6. D. R. Matlack, “Cost Effectiveness of Spinal 
Cord Injury Center Treatment,” prepared by the 
National Paraplegia Foundation for OHDS, 1974. 
[This report provides a CEA of two alternatives 
spinal cord injury treatment methods. ] 

7. Stanford Research Institute, ‘Feasibility and Cost 
Effectiveness of Alternate Long-Term Care Set- 
tings,” prepared for Social and Rehabilitation 
Services, May 1978. [This is a pilot study which is 
designed to identify methods that could be used 
to determine the relative cost-effectiveness of 
various alternate types of long-term care set- 
tings. | 


State-Sponsored Studies 


1. Michigan Department of Education, Division of 
Vocational Rehabilitation, “A  Benefit-Cost 
Analysis of Vocational Rehabilitation Programs 
in the State of Michigan,” 1970. 

2. R. D. Struthers, “A Benefit/Cost Model Devel- 
oped in a State Vocational Rehabilitation Agen- 
cy: Michigan Vocational Rehabilitation,” Jour- 
nal of Rehabilitation Administration, July 1977. 

3. E. Wright, “A Benefit-Cost Analysis of Voca- 
tional Rehabilitation Programs in the State of 
Michigan,” prepared for the Michigan Rehabili- 
tation Service, Lansing, Mich., 1970. 

4. C. M. Grigg, A. G. Holtman, and P. F. Martin, 
“Vocational Rehabilitation for Disabled Public 
Assistance Clients: An Evaluation of Fourteen 
Research and Demonstration Projects,” pre- 
pared for the Institute for Social Research, 
Florida State University, Tallahassee, Fla., 1969. 

5. J. E. Muthard, et al., “The Vocational Rehabili- 
tation of Public Assistance Recipients: A Na- 
tional Survey,” prepared for the Rehabilitation 
Research Institute, University of Florida, 
Gainesville, Fla., 1976. 

6. K. W. Reagles and G. N. Wright, “A Benefit- 
Cost Analysis of the Wood County Project: An 
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Illustrated Lecture,” prepared for the University 
of Wisconsin Regional Rehabilitation Institute, 
1971. 

7. M. Berkowitz, “Benefit Cost Analyses: An Ap- 
plication to a Vocational Rehabilitation Project: 
An Interim Report,” prepared for the Bureau of 
Research, Rutgers University, New Brunswick, 
N.J., May 1976. 

8. C. Cole and R. Dodson, “An Introduction to 
Cost Benefit Analysis of the Vocational Rehabil- 
itation Program: A Model for Use by State 
Agencies,” prepared for the Institute of Urban 
and Regional Development, University of Cali- 
fornia, Berkeley, Calif., October 1972. 

9. F. C. Collignon, “A Working Outline for Cost- 
Benefit Analyses of Vocational Rehabilitation 
Programs,” prepared for the Institute of Urban 
and Regional Development, University of Cali- 
fornia, Berkeley, Calif., November 1971. 

10. D. W. Dunlop, “Benefit/Cost Analysis: An 
Analytical Framework for Vocational Rehabili- 
tation,” prepared for the Michigan Vocational 
Rehabilitation Service, Lansing, Mich., 1969. 

It is generally conceded that these studies possess a 
number of serious limitations. Several of the anal- 
yses, particularly those conducted by State voca- 
tional rehabilitation agencies, suffer from poor meth- 
odological design and inadequate data (52). In addi- 
tion, cost estimates are often imprecise because of the 
difficulty of measuring rehabilitation costs, and 
benefit assessments tend to represent general esti- 
mates because a number of benefits are psychic or in- 
tangible and do not lend themselves to quantification 
(123). 

The 1973 amendments (Public Law 93-112) to the 
1965 vocational rehabilitation legislation (Public Law 
89-333) mandated that Federal officials broaden their 
efforts and work towards achieving the goal of inde- 
pendent living for the severely disabled. Measure- 
ment of the costs and benefits of services to the se- 
verely handicapped is very difficult, though, because 
of the problem of expressing in economic terms the 
worth of independent living. Further, many policy- 
makers emphasize the importance of considering to- 
gether with the costs of rehabilitation programs their 
humanitarian goals, and they argue that a traditional 
cost-benefit framework is inappropriate for this pur- 
pose (123,460,599). 

Currently, OHDS is trying to deal with some of 
these problems and make CBAs more useful for its 
decisionmaking by sponsoring the development of a 
more sophisticated cost-benefit model. At the same 
time, however, several officials emphasized the dan- 
ger of using CBA without understanding its limita- 
tions (479). Many seem to agree with John Noble’s 
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observation made in 1977 that “the state-of-the-art 
needs substantial upgrading before CBAs can be 
taken seriously as a guide to priority-setting in the 
field of rehabilitation” (461). 


Veterans Administration 


VA operates the largest centrally directed hospital 
and clinic system in the United States. VA also is ex- 
tensively involved in medical and health services re- 
search. Apart from one current study of the cost-ef- 
fectiveness of hospice care, however, VA has not 
been involved in conducting cost-effectiveness or 
cost-benefit studies. Health systems research officials 
intend to study the results of the hospice study, “An 
Evaluation of the Wadsworth Palliation Treatment 
Programs,” which is to be completed in 1983, in 
order to assess the feasibility of using CEA (184). 

Officials within VA are considering intensifying 
VA's efforts in the evaluation of health care tech- 
nology and are therefore interested in exploring the 
cost-benefit methodology. VA's new health services 
R&D director, Dr. Richard J. Green, has expressed a 
special interest in the use of evaluation techniques 
such as CEA in the examination of health care issues. 
A few of the areas in which VA hopes to employ 
CEA techniques in the future are rehabilitation medi- 
cine, alternative models of care, extended care pro- 
grams, and contracted services. An area that VA 
hopes to focus more attention on in the future is pre- 
ventive care and preventive care packages for veter- 
ans. It is uncertain at this point whether CEA will 
play a role in the evaluation and planning in this 
area, but it seems clear that there is great interest in 
its use. 


State and Local Governments and 
Nongovernmental Organizations 


Although a few State and local governments and 
nongovernmental organizations have had experience 
with cost-effectiveness and cost-benefit studies, they 
appear to use such analyses only rarely. This is not 
surprising for at least two reasons. First, these groups 
traditionally devote far less funds and staff to evalua- 
tion than the Federal Government does. And second, 
State, local, or regional CEA/CBAs tend to be expen- 
sive because the necessary data are generally difficult 
to obtain. Where State and local CEA/CBAs have 
been conducted, their performance has usually re- 
flected individual staff interest in CEA/CBA tech- 
niques. Perhaps, the one major exception to this gen- 
eralization lies in the area of rehabilitation, where 
many State and local governments have followed the 


Federal Government's lead in using cost-benefit 
studies to justify investment in vocational training.*® 

As the following lists of State and local studies 
show, apart from vocational rehabilitation studies, 
most State and local CBAs have been conducted in 
Massachusetts and New York (19). 


State Government Studies 


1. Massachusetts Department of Public Health, 
“Cost-Benefit Analysis of New Born Screening for 
Metabolic Disorders, N. Eng. J. Med., 291:1414, 
1974. 

2. Massachusetts Department of Public Health (M. 
E. Farber and S. N. Finkelstein), “A Cost Benefit 
Analysis of A Mandatory Premarital Rubella-An- 
tibody Screening Program,” N. Eng. J. Med., 
Apr. 12, 1979. 

3. D.S. Shepard, and M. Thompson, “The Econom- 
ics of Prevention: The Method of Cost-Effective- 
ness Analysis,” prepared for the Office of State 
Health Planning, Boston, Mass., July 1977, re- 
vised and published as “First Principles of Cost- 
Effectiveness Analysis in Health,” Public Health 
Reports, November/December 1979. 

4. A. Rogers and P. M. Bloomburgh, “Cost-Effec- 
tiveness Under Medicaid,” Dec. 31, 1979, pre- 
pared by staff of the Massachusetts Department 
of Public Welfare for the Health Care Financing 
Administration. [HCFA terminated this project 
before its scheduled completion. ] 

5. E. L. Hannany and J. K. Graham, “A Cost-Bene- 
fit Study of a Hypertension and Screening and 
Treatment Program at the Work Setting,” In- 
quiry, 4(4):345, December 1978, prepared for the 
New York State Department of Health, Bureau of 
Disease Control. 

6. J. K. Paperfuss and B. C. Ejeldshed, “Cost-Benefit 
Study of Selected Interventions in Control and 
Prevention of Tuberculosis in the State of Michi- 
gan,” prepared by the Michigan Division of 
Health for the Center for Disease Control, De- 
cember 1969. 


Local Government Studies 


1. A. Leslie, “A Benefit/Cost Analysis of New York 
City Heroin Addiction Problems and Programs, 
1971,” in Analysis of Urban Health Problems, 
edited by I. Levinson and J. Weiss, New York: 
Spectrum Publications, Inc., 1976. 

2. M. L. Ingbar, “Data System To Evaluate Cost-Ef- 
fectiveness of Ambulatory Health Services to the 


"See the section of this appendix on the Office of Human Development 


Services (p. 154) for a discussion of these State and local studies. 
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Poor in Cambridge, Massachusetts,” 1971, pre- 
pared by staff of the Department of Health, Hos- 
pitals, and Welfare, Cambridge, Mass. 


Nongovernmental Studies 


Among nongovernmental agencies, cost-effective- 
ness and cost-benefit studies also have been infre- 
quent. Further, as the following list shows, those 
studies that have been conducted have often been 
funded by the Federal Government. 

1. D. R. Matlack, “Cost-Effectiveness of Spinal 
Cord Injury Center Treatment,” prepared by the 
National Paraplegia Foundation for the Office of 
Human Development Services, HEW, 1974. 

2. A. Zuvekas, “Cost-Effectiveness of Community 
Health Centers,” prepared by the National Asso- 
ciation of Community Health Centers for the Na- 
tional Center for Health Services Research, 1979. 
[This study provides a cost-effectiveness method- 
ology for evaluation of CHCs.] 


. M. M. Kristein and S. Jonas, “A Cost-Effective- 


ness Manual for HSA Planning. Prepared by the 
American Health Foundation for the National 
Center for Health Services Research, 1980. [This 
study will provide an HSA cost-effectiveness 
manual with particular emphasis on health pro- 
motion areas. ] 


. M.M. Kristein and C.B. Arnold, “Mammograph- 


ic Screening for Breast Cancer: The Economic 
Analysis,” paper presented at the American Pub- 
lic Health Association, Oct. 17, 1978. 


. M. M. Kristein, ‘“Cost-Effectiveness of Various 


Smoking Cessation Methods,” prepared by the 
American Health Foundation, 1978. 


. M. M. Kristein, “The Economics of Secondary 


Prevention: Screening for Disease: An Example 
From Colo-Rectal Cancer,” prepared by the 
American Health Foundation for the National 
Cancer Institute, 1978. 
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Introduction 


Formal CEA and CBA are applied to a wide range 
of topics in a number of diverse areas. A CEA or 
CBA can be performed by a single analyst or by a 
dozen or more. An analysis can take a few months to 
complete or may require more than a year. The size 
of the problem, the availability and quality of data, 
the complexity of the issues involved, and the pres- 
ence of the right mix of professionals combine to ex- 
ert considerable influence on the cost, the quality, the 
usefulness, and the credibility of the analysis. 

A major focus of The Implications of Cost-Effec- 
tiveness Analysis of Medical Technology project was 
on the feasibility and implications of using CEAs 
and/or CBAs in the health care decisionmaking proc- 
ess. An important component of that focus are the di- 
rect resource costs of performing these types of anal- 
yses. The cost of producing a CEA or CBA will sig- 
nificantly influence the use of the methodologies. 
What are the resource needs, the problems likely to 
be encountered, and the time needed to produce 
quality analyses? What variables influence the re- 
sources used or needed to perform various CEAs or 
CBAs in the health care system? The feasibility and 
potential impacts of using CEA or CBA in the health 
care system are directly tied to such questions. To 
identify and discuss the range of answers, OTA con- 
ducted a survey of the resource costs of CEA/CBA. 


Method 


The survey instrument OTA used was a question- 
naire designed to explore the types and amounts of 
resources required to perform a CEA and/or CBA of 
various health care technologies. (See the addendum 
to this appendix for a copy of the questionnaire.) 
Two types of information were desired: first, the 
resource costs of performing actual CEAs or CBAs 
that have been published in the literature; and sec- 
ond, the resources the sample group, comprised of 
the individuals who performed those analyses, felt 
would be required to staff a hypothetical research 
team responsible for conducting CEAs or CBAs on 
medical technologies on a regular and continuing ba- 
sis. In essence, the survey sought a listing of re- 
sources that had been used for actual studies and esti- 
mates of resources needed to perform CEAs and 
CBAs on a routine basis. 

The sample population was chosen on the basis of 
an analysis of the health care literature and discus- 
sions with analysts in the health care area. Thirty- 
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five studies were selected by this process; no attempt 
was made to randomize the selection process. The 
survey was sent to analysts identified as having done 
“quality” work or whose studies were cited frequent- 
ly in the health care literature. Not necessarily all 
quality studies were identified and selected by this 
process. 

Twenty-two responses were returned. Eighteen re- 
spondents answered the questions pertaining to the 
resources actually used in performing the published 
analyses. Twenty-two respondents answered the 
questions related to the resource needs of the hypo- 
thetical CEA/CBA team described in the survey. 


Results 
Resources Used in Published CEAs and CBAs 


The number of professional-level people directly 
involved in a single analysis ranged from 1 to 10 per 
study team, with the mode at 3 and the mean at 3.7. 
There was insufficient information to determine the 
degree of effort, or percentage of time, that the vari- 
ous professionals devoted to the studies. The re- 
sponses were so varied and wide ranging that it was 
difficult to characterize the amount of time that, say, 
a physician or economist spent on a given analysis. 
The amount of full-time effort devoted to the studies 
ranged from as little as a single day to as much as a 
full year. 

One trend that did emerge from the survey was the 
use of physicians on the study teams. Only one study 
did not have a physician directly involved in the 
analysis; the remaining studies had at least one physi- 
cian, and several had two or more physicians, as part 
of their research group. The professions that were 
used in performing actual CEAs or CBAs on health 
care topics are summarized in table C-1. 

The survey also attempted to identify the informa- 
tion used by the analysts. Their responses are sum- 
marized in table C-2. The types of data that the vari- 
ous analysts used tended to be study-specific. Re- 
sponses included data on investment and operating 
costs, health education program costs, drug costs, 
screening costs, travel time to hospital, ambulance 
response time, physician fees, and on a host of other 
cost and benefit variables that cut across the four ma- 
jor categories listed in table C-2. 

The final topic that the respondents were asked to 
address related to the data sources they used and the 
problems they had in obtaining their information. 
Their responses are summarized in table C-3. 
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Table C-1.—Professions Used in Performing Several respondents noted that, although the data 
Health-Related CEA/CBAs may have been free, the time and effort required to 
Met faende sent hone or preter obtain it was expensive. Another problem seemed to 
— q y pe be determining whether the data existed, and if so, 
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Benstin Ease! EV NE is 5 The professions most frequently cited as being es- 
ik tc OL ale a aN lee il sential components of an office designed to perform 
puteranalysiSnt. Wey anki ee 3 . 
Computer programing............ 2 CEAs and CBAs of health-care-related technologies 
Public administration............ 2 on a regular and continuing basis are shown in table 
Business administration ......... 2 C-Ay 
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SOURCE: Office of Technology Assessment. Hospital administration .......... 2 
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Difficult to obtain pNedip ieee is WG wan a ee its Librarian shits ae) SBN Pe See ey ore De 3 
Almost impossible to obtain ...... 6 Accountant i.'. oe s.cae We tao 2 
Had to purchase data............ 0 
Did original research............. 8 Asystems analysis, operations research, and decision analysis. 
Collected from existing sources ... 18 SOURCE: Office of Technology Assessment. 
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to staff the research group. 
The number of studies that this “ideal” office 


companies, etc.) ..,..........: 9 would be able to perform was estimated to be 6 to 10 
Data obtained from nonprofit studies per year with 8 to 12 full-time professionals 
organization...........+.++++: is and the necessary consultants, research assistants, 


Data obtained from other sources 
State government ............. 
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Unpublished reports........... per year. 

Clinical trial information........ The estimated cost per study, not necessarily re- 


SOURCE: Office of Technology Assessment. lated to the aforementioned total office cost esti- 


and support staff present. The estimated cost of the 
research group was between $400,000 and $500,000 
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mates, covered a broad range. The graph in figure 
C-1 summarizes and illustrates the wide range and 
distribution of the cost-per-study estimates. 

Many of the cost estimates were accompanied by 
reservations indicating a close correlation between 
the cost of the study and the extensiveness and severi- 
ty of problems with information needs, the stage of 
the technology’s development, the complexity of the 
problem being addressed, computer needs, and the 


costs to develop the study model. Though not the 
only variables cited, these problems received fre- 
quent mention by the respondents. 


Discussion 


The purpose of this survey was to identify infor- 
mation and personnel variables that might affect the 
“cost” of a CEA/CBA, solicit opinions regarding the 


Figure C-1.—Range of Estimated Costs per CEA/CBA Study 
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4Most of the respondents included a high and low estimate of the cost-per-study. 


SOURCE: Office of Technology Assessment. 
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resources required to do CEA or CBAs on a routine 
basis, and to develop a range of estimates of the re- 
sources used to do published CEAs or CBAs. To a 
large degree, these goals were realized. Because of the 
survey's design, however, no firm conclusions are 
possible. 

The scope of the survey was limited in the sense 
that the process used to select the sample population 
was restricted and arbitrary since the criteria used to 
identify the “quality” CEAs and CBAs were highly 
subjective. The survey sample was limited even fur- 
ther because its focus was restricted to CEAs and 
CBAs done only on health care topics. The survey 
sample may have been further distorted by the char- 
acteristics of the types of issues selected by research- 
ers. Possibly, the CEA and CBA studies done to date 
have been “easy” ones. Analyses of health care topics 
may have been directed at areas that have had the 
most or best data available or that were the most 
highly visible in the public eye. On the other hand, a 
counterargument could be made. Since the field is 
relatively new, perhaps more work was required to 
develop the methods, more analysts were needed to 
perform quality work, and in essence, added effort 
was needed to establish the groundwork that other 
analysts can use. Whichever the case, it is difficult to 
know what effects these variables will have on the 
sample results. 

The survey was broadly focused in that no attempt 
was made to differentiate between the various types 
of analyses. For example, no attempt was made to 
group the studies according to their technological 
focus (i.e., diagnosis v. therapy, procedure v. drug, 
or systems-based technology). Likewise, no attempt 
was made to determine if some analyses were consid- 
ered more complex or sophisticated, or if some had 
more “value” to the research community or to the 
policy process, than others. Although these variables 
are very important, it was beyond the scope of this 
survey exercise to investigate them to the degree nec- 
essary to form estimates or conclusions. 

At best, the survey results suggest a lack of consen- 
sus on most aspects of funding or staffing resources 
required for CEAs or CBAs. A good example of this 
lack of agreement are the estimates of what it might 
cost to perform a single CEA or CBA. In large part, 
however, the many reservations that the respondents 
included regarding the effect of data problems and 
the complexity of the issue(s) being examined on the 
cost of the study would explain the wide range of 
estimates provided. Although there were few sur- 
prises regarding the types of professionals or support 
staff required to do CEAs or CBAs, there was an in- 


teresting change in views in the responses received 
for the questions related to the types of professionals 
actually used to perform specific analyses and those 
related to the types of professionals recommended to 
staff the hypothetical research team. All but one ac- 
tual study included a physician as part of the research 
group, but only seven included an economist. The 
hypothetical research teams, however, leaned more 
heavily toward the inclusion of economists. This ap- 
parent shift toward economists may not indicate any 
real change in the perceived need for physicians, 
though, because several respondents indicated a need 
for physicians and scientists to serve as consultants to 
the hypothetical research group. Thus, some of the 
respondents to the survey listed physicians in another 
category. The shift to more economists, however, 
was not explained. , 

Data needs and problems appeared to be a signifi- 
cant factor for all the respondents. The large range of 
cost-per-study estimates was directly tied to the 
availability and quality of data. Information needs 
were cited much more often than factors such as 
complexity of the problem being studied and stage of 
development of the technology as variables that will 
affect the cost of a given study. To date, however, 
the respondents seem to feel that many of the data 
range from being easy to difficult, but not impossi- 
ble, to obtain. The data also were very inexpensive to 
obtain and were available from public sources. 

The results of OTA’s survey may simply provide a 
look at the types of resources analysts have used to 
perform health-care-related CEAs and CBAs and at 
the types of resources they feel are necessary to per- 
form them on a routine basis. It is interesting to note 
the differences between the resources used and the 
resources that the respondents felt were required to 
do CEA and CBA on a regular basis. Several profes- 
sions that were not used a great deal by the actual 
study teams in the past show up quite frequently on 
the “ideal” research team list. Disciplines such as 
economics, statistics, engineering, computer anal- 
ysis, and computer programing were not frequently 
used in the actual studies, yet were cited several times 
by the respondents as being needed for the hypotheti- 
cal research group. As the health care issues become 
more complex and as information needs become 
more demanding, the range of expertise needed to do 
analyses will broaden, and the costs of performing 
CEAs and CBAs will increase. These resource costs 
may become an important factor to be considered by 
governments and other institutions that must decide 
whether or not to do CEAs or CBAs or how many 
they can do ina given time period. 
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ADDENDUM 
QUESTIONNAIRE USED IN THE 


SURVEY OF THE COSTS OF CEA/CBA 


PART 1 


What is the general cost range that you feel is adequate to perform most 
cost-benefit or cost-effectiveness studies on medical technologies, techniques, 


procedures or systems? 


We realize that the costs will vary according to different factors (e.g., type of 
technology, stage of development). What are the factors that must be considered 
and that are most important in effecting these cost differences, and how would 


they affect the cost of analysis? 


Additional comments: 


Note: This aspect of the survey (Part 1) is very important to us; therefore we 
request that it receive priority should you find yourself short of time; 


however, we do urge you to complete all parts of this survey. 


A. Hypothetical situation: You are asked to staff an office that would be 


responsible for carrying out cost-effectiveness or cost-benefit studies of 
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a range of medical-related technologies, techniques, procedures, and 


Systems. What are the necessary disciplines to carry out this function? 


How many of each? 


ue 6. 
2. Te 
3. 8. 
4, Lt ee 9. 
5. 10. 


What support services or personnel are needed? How many of each? 


t 4, 
Ae 5% 
3. 6. 
PART 2 


The intent of this part of the survey (Part 2) is to obtain a range of 
estimates of resources that have been used to perform cost-effectiveness or 
cost-benefit studies. Since you have either performed or directed such a study, 
we ask for your help in providing us this information. At the bottom of the 


page, we have indicated the study or studies that we wish you to address. 


Perhaps the easiest and quickest way for you to help us would be if you just 
send us the budget breakdown that was developed for the study or Studies listed 
for contract or grant purposes. Should you wish to keep certain parts of it 


confidential, please black them out. 


If you do not have a budget breakdown or would prefer not to send it, we ask 
that you fill out sections I and II below instead. For those of you who are 


sending the budget page, we ask that you also fill out section II only. 
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I. How many professional level people directly contributed to the analysis? 


What types of training or educational backgrounds did these people 
have (what did they consider their professional niche)? Please check 
off the disciplines involved; if more than one person in each 
discipline, please indicate how many. Please indicate level of 


education abesse Mone, rls Us, GUC. 


1. Economics (Specialty ) 


2. Health care professional(s): 


a. Physician(s) __ practicing or academic 
b. ‘Public health. 

ec. Hospital administration _ 

d. Epidemiologist 

e. Scientist(s) _ 


What discipline(s) 


3. Systems analysis _ 
4, Operations research _ 
5. Computer analysis _ 
a. computer programming 
6. Public administration _ 
7. Lawyer 


8. Other 


What level of effort did these people contribute to the study in terms 


of full-time equivalents or person days, weeks, or months devoted to 


ST 
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the analysis? (State type of professional and time spent.) 


Professional 


] 








Time spent on study 








The information needs of the analysis: 


A. 


What types of information did you need for your analysis? Please try 


to be fairly specific (for instance, equipment cost, personnel costs, 


drug costs, cervical cancer deaths per year, disability data, genetic 


disease prevalence data, etc.). 


ae Direct cost(s) data 


Direct benefit(s) data 


Ce Indirect cost data 





Q 
e 


Indirect benefit data 





e. Morbidity data 
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f. 


Ze 


h. 


i. 


Mortality data 


Epidemiological data 


Health services utilization data 


Other data needs 


B. Was the data required for your study: 


Readily available 
Difficult to obtain 


Almost impossible to get in a usable form 


Ge In order to obtain data in a usable form for your purposes, did you 


have to: 

a. Purchase it (buy data tapes, compiled lists, etc.) 
b. Do original research 

ec. Collect from existing sources 


De Was the data 


ae 


Ce 


Free 
Inexpensive to obtain 


Expensive to obtain 


E. Was much or all of the data obtained from: 


ae 


Public sources __ 

1. journals = 

2. books _ 

3. government studies or reports _ 


4. other 
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b. Quasi-public sources, non-profit organizations, foundations, etc. 





ec. Private sources (industry, insurance firms, etc.) 


F. Additional comments or explanations: 
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by The Hastings Center 
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Introduction 


The past two decades have been as hard on science 
as they have been on many other once venerated in- 
stitutions, and for at least one of the same reasons: 
The idea that it consists exclusively of delineation of 
truths about the state of the world has been success- 
fully challenged. 

In some instances, scientific “facts” have been 
revealed to be deliberately falsified. Apart from tar- 
nishing science’s image, falsification has other conse- 
quences. If a cancer researcher fakes results, others 
may spend months or years and large amounts of 
public money attempting to replicate them. If a psy- 
chologist fabricates IQ studies of twins, the data may 
be used in other similar studies and reviews, and the 
cumulated erroneous data may be influential in for- 
mulating a worthless (at best), or even a repressive 
and openly racist (at worst), social policy. 

It is one of the axioms of the scientific enterprise 
that the norms and internal controls of science guar- 
antee that such abuses will ultimately be exposed. 
Even if that is true for flagrant violations, however, it 
is not clear that it is true or even can be true for the 
much more subtle distortions inherent in the daily 
practice of science. These generally involve nothing 
so egregious as deliberate falsification of data, but 
rather much less obvious effects of the researchers’ 
biases and value commitments, many of which are 
unconscious. 

In general, the argument that science’s value- 
freedom is illusory has fallen less harshly on the 
“hard” or natural sciences than on the “soft” or social 
sciences. Economics, however, is almost in a class by 
itself. Since its methods are quantitative almost by 
definition, economics appears to be relatively “hard” 
compared to many of the other social sciences. Partly 
because of this, economics has been one of the least 
criticized social sciences. 


NOTE: This appendix includes nearly all of a report prepared 
by The Hastings Center under contract to OTA. As with all re- 
ports published by OTA, its inclusion does not imply endorsement 
by the members of the Technology Assessment Board or by any 
advisory panels. Although the report focuses on CBA, the discus- 
sions apply in many instances to CEA as well. The report should 
not be considered an exhaustive treatment of ethical issues in- 
volved in the conduct and use of CEA/CBA. 
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In recent years, that situation has begun to change. 
Largely as a result of the “dismal science’s” dismal 
record of forecasting, economics has been subjected 
to increasing criticism. That record has been increas- 
ingly frustrating for citizens and policymakers, both 
of whom must somehow cope with continuing uncer- 
tainties about inflation, unemployment, and other 
economic indicators. Deciding between big bang and 
steady state theories of the universe may ultimately 
have profound effects on our view of mankind, but 
most people are more concerned and more immedi- 
ately affected by theories of economics. The impact 
of economics on government policy and daily life 
underscores the importance of accurate predictions 
and conclusions. 

Among the currently fashionable methods of eco- 
nomics are CEA and CBA. A major text in the field 
of CBA was written by the economist E. J. Mishan, 
who believes economists doing CBA are really asking 
a question similar to that asked by a company ac- 
countant (410). Instead of asking whether the com- 
pany will be better off engaging in one activity as op- 
posed to another, the economist asks whether society 
would be better off by undertaking the project under 
study. In Mishan’s view, CBA applies standard eco- 
nomics to help advance the social welfare. The ra- 
tionale is based on the Pareto improvement,! which 
occurs if some economic arrangement makes people 
better off without making anyone worse off. 

According to Herbert Klarman, a pioneer in using 
CBA in the health field, its use by experts, and the 
idea that it should influence public decisions, is only 
a generation old (317). There are few references to it 
in the literature before 1958. More recently, CBA has 
become a favorite tool among policymakers and has 
been applied in many areas of public life. Along with 
systems analysis, CBA has been widely used by the 
military and for major public works projects.” 

Noting that investment in human capital was a 
popular theme in postwar economics, Mishan sug- 
gests that it was an easy step from investment in 
human productivity to investment in health (410). 


‘Named for the Italian economist and sociologist Vilfredo Pareto 


(1848-1923). “An equilibrim is said to be ‘Pareto-optimal’ if (and only if) 
there is no possible movement from it that could make everyone better off” 
(531). 

2COne famous and very controversial British study, for example, used it to 
help find a site for a third London airport. 
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The rationale is simple: Disease and premature death 
are costly to an economy, so investment in medical 
care can bring significant economic rewards by in- 
creasing human capital. 

An increasing number of cost-benefit and cost- 
effectiveness studies have been done in the health 
field in the last decade, although how influential such 
studies have been in the forging of policy is not clear. 
Rashi Fein has pointed out that the attractiveness of 
economic arguments derives from a belief that eco- 
nomics is value free and objective (194). That makes 
it seem particularly appropriate for fields dealing 
with social policy, such as health care, which are 
fraught with value conflicts and otherwise appear to 
lack rigorous guidelines for decisionmaking. 

Sociologist Duncan MacRae suggests that the his- 
torical development of economics has narrowed the 
discipline’s outlook, making it highly specialized and 
scientific (394). Whereas economists like Adam 
Smith all made contributions to disciplines other 
than their own, today’s economists are either in- 
terested exclusively in economics, or they are in- 
terested in extending economic theory to other fields 
like political science or sociology. 

Thus, MacRae argues, welfare economics is a 
much more closed system than its predecessors, more 
impervious to values that come out of other aspects 
of its practitioners’ lives. According to R. M. Parish, 
this kind of narrow specialization is exactly what 
CBA needs least (475). Good CBA, he says, ‘seems 
to call for a combination of subtle theoretical analy- 
sis, imagination, and a lot of hard, pedestrian slug- 
ging—the capacities to do which are seldom com- 
bined in one individual.” 

The need for economists with diverse capacities 
may be especially great in the area of health. As 
noted by R. N. Grosse, there has been little analysis 
of the relationship between expenditures for and out- 
comes of social programs (265). At the very simplest 
level, we really do not know how medical services af- 
fect our health, in part because “health” is an abstract 
and fuzzy concept. Health programs often really can- 
not be evaluated. We lack agreement on their objec- 
tives and on how to achieve them. We find it hard to 
estimate measures even when we agree on them. We 
cannot sort out cause and effect relationships because 
there are many inputs and outcomes. Finally, we 
have very little information about programs whose 
effects occur over a long time. 

To pick a program goal and then attempt to eval- 
uate how well the program is achieving that goal re- 
quires normative assessments, evaluations, and as- 
sumptions. CBA, however, is simply descriptive. 
Norms, cutoff points, decisions about where to draw 
the line between good and bad outcomes, all are in 
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the mind of the policymaker and need to be imposed 
on the data from outside. Norms are also in the ana- 
lyst’s mind, although analysts’ commitment to the 
disciplinary ethos of quantification and objectivity 
may make them genuinely and sincerely unaware 
that norms are an inevitable part of their description. 

In a book-length critique of CBA, Peter Self asserts 
that CBA depends on value judgments more than 
most types of information (556). Further, some of the 
major figures in economics, including many of that 
field’s most thoughtful practitioners, share sub- 
stantial agreement that economics in general and 
CBA in particular are anything but norm and value 
free. 

This paper lays out major values underlying the 
quantitative techniques of cost-benefit and cost-effec- 
tiveness analysis in the health field. It shows how 
those values can result in ethical problems, and 
makes some suggestions for avoiding the difficulties 
the values engender. Though based on extensive 
reading of both the theoretical and technical litera- 
ture in this field, the discussion is not exhaustive. 
Our brief for this exercise has been well stated by 
Fein, who observed that it is incorrect to suppose that 
measurement is neutral. “Cost-benefit analysis,” he 
says, “is too important to be left to analysts or 
economists” (194). 


Working Definitions of Cost-Benefit and 
Cost-Effectiveness Analysis 


Definitions of CEA and CBA in the literature vary. 
OTA has defined them as follows: 

Cost-Effectiveness Analysis (CEA): An analytical 
technique that compares the costs of a project or of 
alternative projects to the resultant benefits, with 
costs and benefits/effects not expressed by the same 
measure. Costs are usually expressed in dollars, but 
benefits/effects are ordinarily expressed in terms 
such as “lives saved,” “disability avoided,” or “qual- 
ity-adjusted life years (QALYs) saved,” or any other 
relevant objectives. Also, when benefits/effects are 
difficult to express in a common metric, they may be 
presented as an “array.” 

Cost-Benefit Analysis (CBA): An analytical tech- 
nique that compares the costs of a project or techno- 
logical applications to the resultant benefits, with 
both costs and benefits expressed by the same meas- 
ure. This measure is nearly always monetary. 

E. J. Mishan says (410): 

The general question that a cost-benefit analysis 
sets out to answer is whether a number of investment 
projects, A, B, C, etc., should be undertaken and, if 
investible funds are limited, which one, two, or more, 
among these specific projects that would otherwise 


170 ¢ The Implications of Cost-Effectiveness Analysis of Medical Technology 


qualify for admission, should be selected . . . in cost- 

benefit analysis we are concerned with the economy as 

a whole, with the welfare of a defined society, and not 

any smaller part of it. 

Mishan does not concern himself specifically with 
CEA, but his characterization of CBA is broad 
enough to encompass it. 

For Prest and Turvey (487): 

Cost-benefit analysis is a way of setting out the fac- 
tors which need to be taken into account in making 
certain economic choices. Most of the choices to 
which it has been applied involve investment projects 
and decisions—whether or not a particular project is 
worthwhile, which is the best of several alternative 
projects, or when to undertake a particular project. 

This definition, too, is broad enough to encompass 
both CEA and CBA. 

Not surprisingly, therefore, Weinstein and Stason 
observe (627): 

Cost-effectiveness analysis and benefit-cost (or 
cost-benefit) analysis are two related, but quite differ- 
ent, approaches to the assessment of health practices. 
Confusion frequently exists between the two ap- 
proaches, and many analyses that are technically cost- 
effectiveness analyses are often labeled “cost-benefit” 
analyses, and vice-versa. The key distinction is that a 
benefit-cost analysis must value all outcomes in eco- 
nomic (e.g., dollar) terms, including lives or years of 
life and morbidity, whereas a cost-effectiveness anal- 
ysis serves to place priorities on alternative expendi- 
tures without requiring that the dollar value of life 
and health be assessed. 

According to Klarman, since CEA does not require 
valuing life and health in monetary terms, the dilem- 
mas of valuation that arise with CBA can be avoided 
by substituting CEA (317). When we substitute CEA 
for CBA, though, we have lost the tool for setting 
priorities among several fields of activity. 

Richard Layard who edited a major volume on 
CEA and CBA, holds that CEA is useful whenever 
CBA becomes impossible (349). The reason is that 
even if the planned benefit itself cannot be valued 
sensibly, it is still useful to compare the costs of pro- 
viding the same benefit in different ways. Apart from 
providing a value for the benefits, he says, the pro- 
cedures involved in CEA are exactly the same as 
those for CBA. Self, a major critic of the cost-effec- 
tiveness technique, says that cost-effectiveness can 
easily shade into CBA because goals cannot be de- 
scribed with precision (556). 

In general, though, CEA aims to tell policymakers 
the least expensive way of achieving an already 
selected goal, whereas CBA seeks to help them select 
from among a choice of goals. Largely because it is 
considerably more ambitious, CBA presents more 
ethical difficulties than CEA (though the latter is cer- 
tainly not free of them). This paper concentrates 


largely on CBA, but much of what is said is equally 
applicable to CEA. 


implications of the Conflict Between 
Equity and Efficiency 


Among its practitioners, there is substantial agree- 
ment that efficiency is the chief value underpinning 
economics. Those practitioners also freely acknowl- 
edge that since this value is its central one, econom- 
ics—and particularly CBA—gives short shrift to 
another important value: equity. The conflict be- 
tween equity (often expressed by the terms equality, 
justice, fairness) and efficiency is an enduring one in 
both politics and in economics. This conflict is the 
most important and intractable value conflict in CBA 
and is also its most important ethical issue. How, in 
pursuit of the least wasteful way of spending the pub- 
lic dollar for a given commodity, can we also make it 
available to everybody? 

Efficiency.—Economics is grounded in philoso- 
phy.’ Its chief underpinnings came out of the philo- 
sophical school known as utilitarianism. Probably 
the most thoroughgoing attempt to construct a moral 
theory whose chief value is efficiency was made in 
the late 18th century by Jeremy Bentham (49). One of 
the earliest utilitarians, Bentham intended his system 
to apply both to the broad social arena encompassing 
politics and legislation, and to the action of individ- 
uals. The goal of Bentham’s utilitarianism was to 
maximize pleasure and minimize pain for all who 
stood to be affected by an action. 

Bentham’s system presupposes an ability to quan- 
tify pleasures and pains. The process he described for 
judging outcomes required a precise specification of 
the tendencies of actions to produce pleasures or 
pains. All such tendencies are added together, and 
then totaled for all affected individuals until an op- 
timal “balance” of pleasures over pains is reached. 
Bentham’s scheme is sometimes called the “hedonic 
calculus” or the “felicific calculus.” The term “calcu- 
lus” should be taken quite literally, since social deci- 
sionmaking using this method requires straightfor- 
ward calculations—adding and subtracting units of 
pleasure and pain. 

Utilitarians, after Bentham, subscribed to the ba- 
sic idea of maximizing utility as a social goal, but’ 
doubted the feasibility of constructing cardinal meas- 
ures of utility. They sought to achieve the same result 
by employing an ordinal scale. Like Bentham’s, their 
approach still allows for a meaningful notion of max- 
imizing individual welfare by enabling comparisons 


3As Kenneth Boulding has pointed out, Adam Smith was a professor of 
moral philosophy (67). 
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of states of affairs as “better” or “worse,” although it 
is not committed to implausible measurements and 
calculations of pleasures and pains. The resort to or- 
dinal measurement of utility, however, founders on 
the nettlesome problem of interpersonal comparisons 
of utility. Until we work out a common measure for 
one person’s “better” and another’s “worse,” the 
achievement of social welfare will remain an elusive 
goal. 

Equity.—In contrast to the value of efficiency that 
characterizes utilitarian theory, the central value in 
philosophical theories based on the concept of justice 
or fairness is equity. John Rawls’s theory of “justice 
as fairness” (494) offers a compelling alternative to 
the philosophical foundations of CBA. Rawls criti- 
cizes utilitarianism’s willingness to allow the greater 
gains of some to offset the lesser losses of others. In 
theory, all versions of utilitarianism could justify ac- 
tions by a majority that would enhance the major- 
ity’s position at the expense of a minority. Even if it 
could produce a total social welfare based on ordinal 
measures of satisfaction, however, Rawls would still 
object to utilitarianism on the ground that individual 
rights and liberties are liable to get lost in the aggre- 
gating shuffle. Although utilitarianism presents itself 
initially as an individualistic democracy of pleasures 
(“everybody to count for one, nobody for more than 
one’—Bentham), Rawls observes, aggregating the 
desires of each individual into a total system of desire 
actually results in a morally defective impersonality 
that allows some people to be used merely for the 
benefit of others. 

Rawl’s rival conception of equity is unique. 
Whereas some political theorists construe equity to 
mean equality (to everyone an equal share), and 
others treat it as a proportionality of some sort (to 
each according to need, merit, contribution, etc.), 
Rawls begins his notion of distributive justice with a 
preference for equality that is immediately hedged by 
a so-called “difference principle.” This principle per- 
mits significant differences in wealth and social 
status, but only those differences that benefit the 
least advantaged members of society. (This device 
makes interpersonal comparisons of utility unneces- 
sary.) Thus, while it it permissible for a society to 
allow certain kinds of inequalities that raise every- 
one’s standard of living—in fact, Rawls holds that it 
would be irrational to insist on equal shares of fewer 
goods—the difference principle rules out those in- 
equalities that improve the status of the better-off at 
the expense of the worse-off. Once a society adopted 
the difference principle, individuals would be entitled 
to whatever goods or status they could obtain or 
achieve through voluntary transactions on the free 
market, gifts, and so on. 


Two examples will suffice to illustrate the sorts of 
equity problems central to CBA: 1) the different 
ways in which people's lives and preferences are eval- 
uated, and 2) the way in which the future is evaluated 
compared with the present. 


Valuing Lives and Preferences 


The problem of placing different values on differ- 
ent lives is keenly felt and discussed in all the 
theoretical literature on CBA, even by its friendliest 
adherents. This problem is particularly germane to 
the health field and is as old as CBA in that field. The 
17th century estimates of the value of life by Sir 
William Petty illustrate a problem that remains un- 
solved (194). To strengthen his argument about what 
the government might save by transporting people 
out of the city and saving them from the London 
plague, Petty valued people at only £20 each, be- 
cause that was the lowest price for a slave. For his 
largely illustrative and rhetorical purposes, setting 
the lowest possible valuation on labor effectively 
made the point. Petty’s calculations of what the gov- 
ernment might have saved was an underestimate; the 
actual amount would, of course, be much more, 
since not only “slaves” would be transported out of 
the city and saved from the plague, but “more valu- 
able” persons as well. 

In a 19th century sanitation study in Massachu- 
setts, Lemuel Shattuck valued adult men differently 
from women and children (559)—a practice often ad- 
hered to today. Shattuck’s estimates of the benefits of 
better sanitation in Massachusetts involved labor 
costs (calculated at an average of $50 a year) totaling 
almost $5.5 million, to which he added the cost of 
public support of dead laborers’ widows and chil- 
dren. But the latter costs came to only an additional 
estimated $2 million. 


The Issue and the Methods 


Fein puts the whole issue nicely (194): 

Does the measurement of a person's worth in terms 
of his productive contribution really represent our so- 
cial values? I believe that it does not do so. In particu- 
lar, it fails adequately to take account of equity and 
distributional considerations (which many believe to 
be one of the major functions of government). 

Fein points out that taking account of individual 
characteristics such as potential earnings could lead 
society to provide health care services to those with 
the highest potential income and to direct them away 
from those with low earning capacity and less educa- 
tion. Females in such a scheme are at a disadvantage 
compared to males, as are the old compared to the 
young, and blacks compared to whites. In each case, 
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the disadvantaged individuals are the ones with the 
lower incomes. 

Klarman argues that some elements of the contro- 
versy over valuing lives have been settled by recent 
technical developments (317). Whereas 15 years ago 
a wife not employed outside the home did not count 
for very much in a CBA, he notes, recently house- 
wives’ services have been given a monetary value. 
The lives of housewives are now usually figured on 
that basis, though still, of course, at a lower rate than 
the lives of their husbands employed outside the 
home. 

When cost-benefit calculations were begun at 
HEW in the middle 1960's, Agency officials realized 
that the benefits calculation discriminated against 
women and the elderly (265). According to Grosse, 
there were fears among some that too vigorous pur- 
suit of the underlying logic would penalize not only 
health programs for the aged (medicare was just get- 
ting started), but programs for the poor, as well. He 
notes, however, that (265): 

In actual practice, for the particular programs stud- 
ied, these concerns were not critical. The programs for 
cervical and breast cancer looked to be good despite 
their being for women. As for the poor, most of the 
programs considered, especially cervical cancer, syph- 
ilis and tuberculosis, were aimed primarily at them, 
and projects were usually located to serve low-income 
residents. 

By implication at least, this is a way to get around 
some of the distributional problems inherent in CBA: 
If decisionmakers are committed to just distribution 
of resources and are interested in choosing the best 
programs aimed chiefly at the poor, i.e., if equity 
considerations are built in from the start, then the 
problem of equity can be dealt with. This is, of 
course, a big if. 

Mishan believes that although distributional and 
equity considerations do offer grounds for objection 
to CBA, the objection is not a strong one (410). Even 
if a project is cost-beneficial, Mishan notes, it can 
always be rejected on grounds of inadequate atten- 
tion to equity. Writing on the relationship of Chris- 
tian ethics to CBA, Denys Munby agrees, arguing 
that the use of differential income data usually makes 
social decisions easier (416). If one found out that 
only millionaires would be affected by the destruc- 
tion of a particular beauty spot, one might not worry 
very much about its destruction. “What is quite clear 
is that we can make our analysis as egalitarian as we 
want,” he says. What these views require, however, 
is that policymakers be committed to equity—and 
that they act on the basis of that commitment. 

Another major objection to the usual way people 
are valued for purposes of CBA is that focusing on 
income ignores other indices. Mishan acknowledges 


that CBA tends to ignore indicators of social merit, 
but argues that we have no good way now of judging 
their worth (410). 

CEA attempts to avoid the seemingly insoluble 
valuation problem of CBA by simply jettisoning 
much of it. Instead of comparing costs in dollars to 
benefits in dollars, CEA compares costs in dollars to 
benefits expressed in nonmonetary terms (e.g., num- 
bers of deaths averted). As Weinstein and Stason 
note, however, a limitation of both CEA and CBA is 
(627): 

. .. that the benefits and costs to individual mem- 
bers of society need to be aggregated. If the inequita- 
ble distribution of benefits and costs across individu- 
als or groups are of concern, a single cost-effectiveness 
measure will not do. However, as economists are 
wont to argue, over large number of programs and 
practices the inequities are likely to even themselves 
out and, with some exceptions, may reasonably be ig- 
nored. 

Even assuming that some of the aforementioned 
problems could be solved, other problems in valua- 
tion would remain. In comparing the loss of a man’s 
life with that of a woman, one is at least comparing 
two identical types of outcome. As Fein points out, 
however, problems arise in comparing apples-and- 
oranges types of outcomes, for example, lives saved 
in comparison with blindness prevented (194). 
Grosse agrees, noting that when dealing only with 
the cost per death averted, there is no way to com- 
pare or rank diseases that do not usually kill (265). 

CBA has not yet found a common way of measur- 
ing different outputs from different programs so that 
they can be sensibly compared. Finding a common 
unit of measurement is very difficult. We are unable 
to measure units of satisfaction or of happiness gen- 
erated by various government activities, nor are we 
able to compare one person's satisfactions with 
another's. 

One way of dealing with this problem has been by 
using the concept of quality-adjusted life years 
(QALYs). Even the concept’s chief proponents, how- 
ever, concede that the weights it gives to varying 
states of health are subjective, so much so that (627): 

In a cost-effectiveness analysis at the societal level, 
it is therefore essential that a range of possible weights 
be used to reflect the spectrum of individual val- 
ues . . . Most analyses avoid quality-of-life considera- 
tions entirely, quantifying only the change in life ex- 
pectancy. Where the quality-of-life effects are believed 
to be important, however, the credibility of an other- 
wise effective analysis may be jeopardized. Tradeoffs 
involving quality-of-life considerations are made im- 
plicitly by health-care decisionmakers all the time; the 
role of the analysis is to make them explicit. 

The difficulty here, as the authors implicitly ac- 
knowledge when they draw attention to the subjec- 


tive nature of the QALY concept, is that such assess- 
ment reintroduces the problem of determining values 
that the use of CEA is supposed to avoid. 

Mishan’s text contains extensive material on the 
valuation of life and limb (410). Though philoso- 
phers could legitimately disagree with his view, 
Mishan regards saving life as symmetrical with losing 
it, so he bases his techniques almost exclusively on 
established legal methods of compensating victims. 
Self says this is actually an ethical rather than a legal 
position, a position that specifies that involuntary 
losses should be compensated (556). 

Despite repeated expressions of dissatisfaction 
with it, the most common way of calculating the 
worth of a person’s life is by discounting to the pres- 
ent that person’s expected future earnings. Mishan 
discusses alternatives to this conventional way of 
valuing loss of life, one being to perform auxiliary 
calculations that take account of a victim's suffering 
or family’s bereavement (410). He appears to favor 
evaluating the saving of life by referring to what each 
person would be willing to pay or to receive for an 
estimated change in the risk to his or her life. These 
calculations could change over time. Mishan argues 
that one might ultimately want to forgo bereavement 
calculations altogether because of the gradual loosen- 
ing of family ties and the decline of emotional inter- 
dependence. In a wholly impersonal society, any 
member of the community is equally replaceable—in 
such a society there would be no suggestion of com- 
pensating for bereavement. 


Weighting Schemes 


One attempt to improve equity considerations has 
been to use utility rather than dollars, and to assume 
that income has diminishing marginal utility. This 
means each person's income is valued equally up to a 
fixed amount; income over that amount is given less 
importance in the calculations. One obvious difficul- 
ty with such weighting attempts is, as Mishan (410) 
and others (194,556,349) point out, to secure a con- 
sensus on the weighting. And, of course, a weighting 
scheme could still end up making the rich richer and 
the poor poorer. 

Klarman argues that weighting, though not wrong, 
is certainly judgmental (317). To make his point, 
Klarman cites as an example the weighting of earned 
income more heavily than the same amount of mon- 
ey received from public assistance. Such weighting is 
based on the assumption that money earned from 
work is somehow morally better than money re- 
ceived as public largesse. It may be legitimate to use 
such weighting in CBA, but as Klarman states (317): 

What must be recognized is that weights are judg- 
mental, are likely to be arbitrary (at least initially), 
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should be derived in the public arena, and, above all, 

must be clearly stated. 

Layard favors CBA in which valuation is arrived 
at by asking people what they would pay to acquire 
the benefits or avoid the costs (349). Of course, one 
doesn’t usually ask people such questions directly, 
although some economists advocate a questionnaire 
approach. Far more often, values are put on such 
programs indirectly by inferring what people value 
from their behavior, often from what they buy. 

Munby approves attempts at valuation based on 
people’s purchasing behavior (416): 

In principle, the whole analysis is entirely demo- 
cratic; it starts and ends with what people actually 
want, as shown by their actual choices. 

On the other hand, though, as Self argues, willing- 
ness to pay is in some sense related to ability to pay, 
so analyses in which this is used for the purpose of 
valuation are still biased toward the better-off (556). 


The Veatch Critque 


Robert Veatch asserts that valuing lives presup- 
poses that we have decided the question of value to 
whom (603). In some theories of valuing lives, the 
relevant consideration is assumed to be value to 
society or aggregate value. In others, it is value to the 
individual (e.g., self-rating weighting systems), to 
some specific social group, or to the person who pays 
the bills (who might also be the U.S. taxpayer). 

Veatch also points to a problem with the willing- 
ness-to-pay approach for weighting (603): 

At least up to a certain point, the older one be- 
comes, the more valuable life becomes (because, sub- 
ject to willingness to borrow, one is generally willing 
to pay more to avoid death risks since one has a 
higher income level). On the other hand, in human 
capital terms, the older one becomes, the less valuable 
life is, to the point that at retirement (at least for males 
who tend not to be involved in non-monetized domes- 
tic labor), life is “worthless”. . . Any formula based 
on personal estimates of how much one would pay to 
avoid a certain risk of death accepts the status quo dis- 
tribution of incomes in our society. Accepting it may 
be efficient—it may be utility maximizing—yet I am 
still convinced it is not just and it is not right. 

Veatch argues that the Kantian maxim that the in- 
dividual is to be treated as an end, and never only as 
a means, prohibits policies in which one individual or 
group benefits at the expense of another, even if the 
net benefits far outweigh the harms. Many econo- 
mists have dealt with this and similar arguments by a 
pragmatic counterargument—that we are in fact 
making precisely such invidious comparisons with 
current policies arrived at on intuitional or other 
“nonscientific” grounds. By this argument, an impor- 
tant advantage of valuing lives for purposes of CBA 
is that it brings us face-to-face with the fact that, even 
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in a supposedly egalitarian society like the United 
States, we value different people's lives differently. 

As Veatch says, however (603): 

If it is conceptually unsound to equate the real value 
of a life with the value of the labor that life will pro- 
duce, it is also unsound to conclude that there is any 
necessary relationship between what I would pay for a 
program to add years of life, and what society ought 
to pay for that program. 

Willingness-to-pay approaches favor those with 
greater ability to pay. Even willingness-to-pay meas- 
ures that concentrate on percentage of income rather 
than absolute dollars really do not solve the problem, 
because selection of an acceptable percentage of in- 
come also has a very strong relationship to income 
level. 

Finally, willingness-to-pay measures fail to tell us 
anything about people’s attitudes toward programs 
in which they have no particular stake, like prenatal 
diagnosis for women who have no plans to bear chil- 
dren and are not close to anybody who has such 
plans. Sheer self-interest dictates that one should be 
willing to pay for programs in which one is or may be 
personally involved. As Veatch notes (603): 

One might have a self interest in avoiding the wit- 
nessing of suffering around him, but this does not 
seem to be the real moral basis of a commitment to 
help programs designed to affect a condition one is 
virtually certain not to have. 

Veatch here ignores the argument (based on CBA!) 
that such programs will save taxpayers money (e.g., 
by reducing the cost of institutions for the retarded). 

Veatch identifies four different assumptions that 
can be used as the basis for valuing lives: 1) all lives 
are of equal value, 2) the prime of life is most worth- 
while (because it is most productive), 3) youth is 
most worthwhile (because young people have more 
life ahead of them), 4) old age is most worthwhile 
(because older people are wiser?). Still Veatch says 
(603): 

.. all have an equal claim to the health care 
needed to provide a level of health equal, insofar as 
possible, with other people's health . . . it will neces- 
sarily have to give way to other claims upon occasion, 
but it should be the decisive starting point for evaluat- 
ing alternative public policy. 

Thus, social usefulness, willingness-to-pay, future 
productivity, and other such measures are and ought 
to be irrelevant. 

CBA is all right for some things, Veatch says, like 
telling us that some diseases kill people in the prime 
of life. Whenever policy choices conflict with indi- 
vidual rights and justice, however, Veatch wishes to 
promote justice “even if that means lower aggregate 
indicators of utility” (603). Younger people should 
receive a higher priority than others, because “an 


essential part of the egalitarian principle of justice is 
that there should be, insofar as possible, an equal op- 
portunity to live to the same age as others.” There 
should also be equality of opportunity to live life 
without suffering, Veatch believes. But here the 
policy objective should not be to achieve a maximum 
reduction in suffering (because that is an aggregate 
measure), but rather to reduce those conditions that 
produce the greatest suffering in the individuals who 
have them. Veatch believes that sometimes the needs 
of the poor may have to be placed ahead of those 
who may die relatively early. He also favors ex- 
cluding or giving low priority to deaths for which the 
victim is personally responsible (e.g., death from 
lung cancer caused by cigarette smoking). According 
to Veatch, involuntary conditions should have 
higher priority than voluntary ones. 


Discounting 


Another important distributional problem that re- 
quires at least a brief discussion is selection of the dis- 
count rate and the related problem of comparing the 
future with the present, given inflation and the fact 
that, in general, people prefer benefits now to later. 
Self traces the problem, in part, to the Pareto princi- 
ple itself, which he believes is ‘strongly biased 
towards the status quo” (556). 

Layard (349) points out, and other analysts (317, 
194) agree, that the question of the welfare of future 
generations is simply ignored in most CBAs. Layard 
believes most economists would argue that projects 
should be judged exclusively in terms of their effects 
on the welfare of those now living. He notes (349): 

If one takes the alternative view that cost-benefit 
analysis aims to throw light on what is right, it is diffi- 
cult to think of any ethical justification for ignoring 
future generations. A practical argument is sometimes 
put forward for ignoring them—that we cannot know 
their preferences. However, there are many items (like 
life) where we do not know how they are valued by 
present generations, and many (like bread) that we 
can be fairly sure what future generations will feel. 

The discount rate, which takes into account what a 
dollar invested today would earn in interest if it were 
not being spent on health care, tends to devalue the 
future. MacRae says economists’ evaluations of the 
future may be wrong (394). Rawls has even suggested 
a zero discount rate to promote justice among gener- 
ations (494). In his discussion of the discount rate, 
Mishan describes the political constraints on deci- 
sions about which discount rates to use (410). Klar- 
man notes that Federal agencies have traditionally 
used a wide range of discount rates, usually without 
giving any kind of justification (317). The conse- 


quences of choosing a high or a low rate, he says, are 
clear: Low rates favor projects or programs whose 
benefits accrue in the distant future. When the proj- 
ect is short-lived, the choice of a discount rate is mi- 
nor. Nevertheless, Klarman says, there may even be 
some merit in employing a single number for all pub- 
lic human investment projects. 


Conclusions 


The distributional issues discussed above present 
both philosophical and practical problems for econo- 
mists. According to Layard, most economists would 
argue that income should be redistributed by direct 
cash transfer rather than by juggling choices of public 
projects (349). But Layard believes that if we are 
reasonably certain that the cash will not be so distrib- 
uted, then the poor person’s dollar ought to be val- 
ued more highly than the rich person’s. 

As he sees it, the problem is less one of equity than 
it is of knowing which constraints should be taken as 
given and figured into the analysis. If a Government 
agency knows for certain that cash will not be redis- 
tributed, even if it should be, then the agency should 
allow for distributional factors when it evaluates a 
project. It should not allow for those factors if it can 
ensure that redistribution will be achieved by some 
more appropriate method. In practice, however, a 
Government agency cannot know this. Until this is 
settled, it may be impossible to rationally appraise a 
project. Privately employed economists, too, are free 
to incorporate distributional measures (e.g., no-cost 
transportation to neighborhood health centers for the 
elderly ) into their equations. 

According to Layard (349): 

A decision-maker should choose from his available 
set of actions the ones which maximize social welfare, 
subject to all the constraints over which he has no con- 
trol. 

It should be noted, however, that this approach 
allows economists enormous latitude without requir- 
ing that they incorporate distributional considera- 
tions. It can also be argued that such an approach 
completely subverts the purpose of CBA, which at- 
tempts to free the policy process from the individual 
beliefs and biases of the policymaker. 

To summarize, the traditional approach to CBA 
excludes formal consideration of distributional ef- 
fects such as equity and fairness. Since economists 
disagree about how to solve this problem, equity 
considerations are likely to continue to be underem- 
phasized in practice. Mishan, for one, believes that 
the situation is unlikely to change in the foreseeable 
future (410). 
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Implications of Data Problems 


After distributional/equity issues, the criticism 
most often made of CBA is that the data employed 
are simply not reliable. This criticism appears to be 
legitimate to an undetermined, possibly large, extent. 
It is a very serious criticism, chiefly because it con- 
flicts with the quantitative and scientific trappings of 
CBA. 

The general problem here, as Parish points out, is 
that CBA is a form of modeling, and, like all other at- 
tempts at modeling, expresses only certain aspects 
(and those perhaps not wholly accurately) of a very 
complex world (475). CBA is “irretrievably” second- 
best; that is, CBA seeks to explore the consequences 
of particular actions in the imperfect economy as it 
exists and in which the insights of theories applicable 
to a first-best world may be misleading. According to 
Parish, “Its practitioners are frequently tempted to 
simplify their problem by making first-best assump- 
tions .. .” (475). 


The Problem of Defining Outcomes and Goals 


Criticisms of data are made on a number of dif- 
ferent grounds. Most obvious are the simple inade- 
quacies of our current reporting and data systems. In 
a very real sense, of course, such inadequacies are 
technical problems that can be improved once they 
are identified and someone in a position of power 
decides to deal with them and improve the system. 

Fein, however, has identified a number of much 
more difficult data problems (194). One concerns the 
measurement of outcomes which are conceptually 
amorphous (e.g., higher levels of health) and to 
which many factors contribute (e.g., housing, in- 
come, nutrition, environment, and medical care of 
all kinds). In addition, there are many factors whose 
relative contribution may differ for different persons, 
and whose proportional importance is largely un- 
known. Another problem is how to measure the 
goals of a health program that exist on a continuum. 
It is more difficult to measure continuous states with 
a wide range of effect, like pain or impairment of 
functional ability, than to measure discontinuous 
states like life and death. Furthermore, difficulties in 
measurement create a bias in favor of programs that 
have easily measurable goals. 

Most discussions of CBA mention the selection of 
goals and objectives as a problem. Many critics and 
even some proponents of CBA feel that the goals are 
often vague and nonspecific (e.g., “health”). Kenneth 
Boulding points out that we often agree on major 
goals (67). We disagree on how to reach the goals, 
and in some cases, on what the alternative ap- 
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proaches might be. Mishan, for example, criticizes 
Klarman’s study of syphilis by pointing out that the 
calculations were based on the goal of eradicating the 
disease (410). There was no comparison with a defin- 
ed control program to reduce the disease to some pre- 
specified level. 

Ironically, Klarman himself is quite aware of the 
difficulty (317). He believes the costs of a particular 
health program can be estimated, but that the field’s 
chief difficulty arises in formulating the contents and 
expected outcomes of programs before estimating 
their cost. In a study of the end-stage renal disease 
program, for example, CEA pointed to the superiori- 
ty of kidney transplantation, with dialysis only for 
initial and backup support. Klarman believes that if a 
CBA had been performed, the shortage of suitable 
kidneys and the relative ease with which dialysis fa- 
cilities could be expanded might have yielded a 
higher net benefit value for dialysis. 

Economists may also make naive assumptions 
about the practicality of stated goals and outcomes, 
Klarman says (317). At least in the early days of 
CBA, he notes, there was a tendency by economists 
to attribute greater efficacy to medical care than we 
now believe is warranted. 


The Problem of Uncertainty About the Future 


The whole area of uncertainty about the future 
—guesswork about costs and benefits of technologi- 
cal innovations and unanticipated shifts in demand— 
continues to plague CBA. According to Mishan 
(410): 

The problem of how to make decisions in any situa- 
tion where the past affords little or any guidance is not 
one that can be satisfactorily resolved either by logic 
or empiricism, and what moves have been formulated 
are either of limited application or of no practical 
value. 

To cope with uncertainty, one can figure an arbi- 
trary cutoff period, adjust the discount rates, or sim- 
ply pick a subjective probability. A number of recent 
articles (627,564) recommend using sensitivity anal- 
ysis (627): 

. . . In this method, the most uncertain features and 
assumptions in the cost-effectiveness calculation are 
varied one at a time over the range of possible values. 

If the basic conclusions do not change when a particu- 

lar feature or assumption is varied, confidence in the 

conclusion is increased. If, instead, the basic conclu- 
sions are sensitive to variations in a particular feature 
or assumption, further research to learn more about 

that feature may be especially valuable. . . 

. . . Examples of sensitivity analyses that are often 
useful are to vary the estimates of the degree of clini- 
cal efficacy of the procedure in question, to vary the 
weights assigned to various quality levels in comput- 


ing quality-adjusted life expectancy, and to test a 
range of discount rates, say, from 0 to 10 percent per 
year. 


The Problem of Using Proxy Goals 
and Measures 


Another problem comes from the use of proxy 
goals and measures that may bias the number and va- 
riety of options for action. Fein’s cites as an example 
measuring the health of children by counting school 
days missed (194). A program designed to improve 
children’s health so that they miss fewer days, he 
argues, is different from one that focuses so heavily 
on reducing days of absence that its success results in 
sick children’s being sent to school. 

In fact, some argue that one of the deficiencies of 
CBA is that it is a complete exercise in proxy meas- 
ures, since, as Fein himself points out, monetary 
benefits are only a part (and in some cases a small 
part) of all benefits, and they do not represent a 
stable or constant fraction of all benefits. This prob- 
lem has been obvious at least since Lemuel Shattuck’s 
1850 sanitation study in Massachusetts (559). By 
spending $3,000 on sanitation planning and technical 
assistance, he argued, the State would ultimately 
gain at least $7.5 million in productivity for the 6,000 
unnecessary deaths averted by efficient sanitation. 
But Shattuck also understood, and said, that eco- 
nomic benefits were not the only ones that sanitation 
measures would yield. In addition, such measures 
would increase public happiness (194). CEA is an 
attempt to sidestep such problems, particularly 
through the concept of QALYs (627,563). 

For Fein, the choice of data is very important be- 
cause budget officials will tend to focus on those out- 
comes that have numbers, or more specifically, eco- 
nomic values, attached to them (194). Programs that 
affect future productivity thus come to be overvalued 
because they can be quantified, and programs that 
relieve pain and suffering but do not affect produc- 
tivity come to be undervalued. According to Fein 
(194): 

It may, indeed, be that programmes addressed to 
disabling conditions and to disease involving mortali- 
ty, rather than to conditions that do not remove the 
person from economic activity, should be favored. 
That conclusion, however, should not be reached pri- 
marily because some things can be measured while 
others cannot. The analyst may discount the nature of 
the difficulty and the likelihood that this might occur, 
believing that his description of the items (particular- 
ly, benefits) that cannot be measured will suffice to 
alert the decision-maker to the inadequacy of the 
numbers. I suggest, however, that the analyst may 
underestimate the problem. He would do well to con- 
sider how compelling numbers are to finance officials 
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and how high a rate of discount is applied to words, 
however well-turned the phrases may be. 
In an article on the Christian ethics of CBA, Denys 
Munby argues that the technique attempts (416): 
... precisely to take account of all the “social 
costs” that social reformers have (rightly) accused the 
pure market economies of ignoring . . . . Money is 
merely an abstract measuring rod; there is no other so 
comprehensive. It is not the use of money symbols 
that can be criticized, but the actual valuations have 
to be examined to see whether they correspond to the 
way we think things should be valued. 


The Problem of Deciding on Inclusions and 
Exclusions in an Analysis 


Klarman points out the great difficulty in deciding 
which factors to include and which to exclude in an 
analysis, and suggests that this problem is particular- 
ly acute in the health field (317). Other analysts 
agree. Grosse notes that the HEW study of the costs 
of automobile accidents and their prevention did not 
calculate the cost of seat belts (265), but he does not 
specify why. That particular omission seems very 
puzzling. Similarly not figured into the analysis was 
the possibility that lower auto insurance rates for 
everybody could result from a successful injury 
prevention program. 

In genetic screening, where many CBAs have been 
performed, similar puzzlements crop up. A number 
of cost-benefit studies have been conducted on new- 
born screening for phenylketonuria (PKU), a very 
rare metabolic disorder leading to severe mental re- 
tardation (79). Mental retardation can be partially 
prevented by putting babies with PKU on a special 
diet shortly after birth. On paper, PKU-screening 
programs for newborns have been demonstrated 
over and over again to save society money by reduc- 
ing the number of retarded people who need to be in- 
stitutionalized. 

But not one of the analyses of PKU-screening pro- 
grams has included in its calculations a major long- 
term cost—that of finding and putting back on the 
special diet during pregnancy young women with 
PKU whose children will otherwise be severely re- 
tarded. Formerly, women with PKU lived in institu- 
tions and did not reproduce. As a result of the special 
diet initiated after birth, today many women who 
were born with the disorder lead normal lives and 
have children. . 

The cost of a 20-year or longer followup for these 
women is enormous, and yet it is justified on both 
practical and ethical grounds. Having saved young 
women born with PKU from severe retardation, the 
state wants to prevent retardation in their offspring 
for which it might otherwise have to bear financial 


responsibility. In addition, it can certainly be argued 
that Government also has a moral obligation, once a 
public policy has been instituted, to help those people 
who are affected deal with new problems that arise in 
their lives as a result of State intervention. 


The Problem of Infinite Externalities 


Boulding points out that almost everything we do 
turns out differently from what we expect; both the 
bad and good are often unintentional (67). So, it is 
not surprising that a major problem with assessing 
cost and benefits are so-called externalities, otherwise 
known as external effects, neighborhood effects, side 
effects, spillover effects, or spillovers. Externalities 
include such things as the effect of building a road on 
esthetic sensibilities, on noise and pollution, on loss 
of life as a result of increased traffic accidents, and so 
forth. 

What characterizes all these effects, Mishan says, 
is that they are unintentional and not subject to con- 
trol by the people who experience them (at least not 
unless there is some way such people can spend mon- 
ey to avoid them) (410). Mishan suggests that the 
number of external effects in the real world is virtual- 
ly unlimited. He believes society is obliged to limit 
them for the economist. 

The number of effects that can be internalized into 
the pricing mechanism, Mishan says, is limited. Ac- 
cording to Mishan, the costs of spillovers such as 
traffic noises, pollution, radioactive waste, and 
diseases of the nerves, heart, and stomach caused by 
high-tension living cannot be internalized. Internal- 
ization would require that the potential victim of the 
spillover have a legal property right to some measure 
of quiet or clean air, freedom from tension, and so 
forth. For that right to be enforceable, it would be 
necessary to delineate a territory around each indi- 
vidual that belongs solely to that person, so that an 
intrusion subject to legal compensation procedures 
could be identified. Since it is impossible to create a 
market, even an artificial one, that would make pric- 
ing them possible, such spillovers must ultimately be 
evaluated by the victims’ subjective estimate. 

Deriving his argument from John Stuart Mill, a 
19th century British disciple of Bentham, Mishan 
thinks compensatory sums should actually be paid to 
victims of spillovers. He rejects the social engineering 
approach to spillovers which seeks to formulate tol- 
erance levels for society (410): 

If the liberal economist rejects social engineering 
norms such as “tolerance level,” it is not merely be- 
cause the choice of such a level for society is necessar- 
ily arbitrary, but because the adoption of such toler- 
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ance norms on behalf of all members of society runs 
counter to the doctrine that each man is deemed to be 
the best judge of his interests, particularly in matters 
that affect him intimately. 
A good portion of Mishan’s book is devoted to a dis- 
cussion of possible compensation and legal liability 
in such situations. 

Prest and Turvey argue that CBA is not relevant to 
decisions on large investments because such large in- 
vestments spill over so much that they may end up 
altering the whole economy (487). The example they 
use is the building of a dam in an underdeveloped 
country. Prest and Turvey’s argument, however, 
might apply equally well to health care. 

Grosse recounts that HEW’s study of the possibil- 
ities for comprehensive health care programs for 
young children looked good enough that Congress 
became very interested (265). It was clear, however, 
that if children who then lacked access to good med- 
ical care were to be provided with conventional pedi- 
atric services, an acute shortage of doctors would 
result. Ways had to be found to use medical man- 
power more efficiently. Thus, the Social Security 
Amendments of 1967, which provided for early case 
finding and treatment of birth defects and chronic 
conditions in children, also provided for a research 
and demonstration program to train and use physi- 
cians’ assistants. 

Training physicians’ assistants is an example of a 
side effect, because such training was not counted as 
a cost in the original HEW study. Nor was it antici- 
pated how a CBA of this sort, translated by law into 
public policy, can influence the practice of medicine. 
This analysis led fairly directly to an increase in in- 
terest in physicians’ assistants, which may change the 
whole hierarchical structure of medicine in the 
United States. It is not clear whether such a develop- 
ment should be counted as a cost or a benefit, or 
possibly a bit of each. Ordinarily, however, a mas- 
sive (usually entirely unforeseen) ultimate effect such 
as this simply does not figure into the calculations. 


The Problem of Pricing the Unpriceable 


Finally, there is the stubborn problem of valuing 
intangibles, or pricing the unpriceable. In discussing 
intangible costs like pain, discomfort, and grief, 
Klarman notes that one difficulty in valuing them is 
that they accrue partly to patients, but also partly to 
their friends, relatives, and to society (317). One way 
of valuing such intangibles is to ask what an indi- 
vidual would be willing to pay to avoid them. But 
Klarman believes the measurement problems here are 
major. Although he describes a number of different 
ways such intangibles have been valued, he is not 
very enthusiastic about any. Mishan points out that 


Klarman himself, when he did his 1965 calculations 
on syphilis, attributed more than 40 percent of the 
final benefit to “reduction of stigma,” which was val- 
ued (essentially arbitrarily) at either 1 or 0.5 percent 
of earnings subsequent to the discovery of syphilis 
(410). 

Parish, along with most other commentators, also 
discusses the difficulty of valuing intangibles (475). It 
is true, says Parish, that such valuations are often im- 
possible. Further, he notes (475): 

. ingenious, indirect means are often unconvinc- 
ing and tend to discredit benefit-cost analysis. Also 
objectionable is the practice of finely calculating the 
more easily quantifiable elements while ignoring or 
drawing insufficient attention to the intangible ones. 
Indeed, this may be a more heinous event than to at- 
tempt to heroically quantify the unquantifiable. 

Because intangibles are not sold in markets where 
consumers are forced to reveal their preferences, says 
Veatch, the willingness-to-pay principle is difficult to 
apply to them (603). 


The Fictional “Facts” 


All of these quite genuine measurement difficulties 
mean that CBA can never really tell the unvarnished 
truth, and that it is therefore at bottom a kind of sys- 
tematic misrepresentation of the world it purports to 
measure. Self goes so far as to accuse cost-benefit 
analysts of (556): 

. . unwittingly or not, playing a confidence trick 
with the symbols of monetary exchange. Of course the 
theoretical welfare economist is not a confidence 
trickster, in fact he is often high-minded, but he is 
committed to the discovery of some ideal (‘optimum’) 
set of economic conditions that transcend the ordinar- 
ily market economy, with the aid of which he can 
measure intrinsic value. 

Many commentators believe that fact saddles econo- 
mists with a heavy moral responsibility, and speak of 
it in exactly such terms. Mishan, for instance, ulti- 
mately characterizes CBA as horse-and-rabbit stew, 
the rabbit representing costs and benefits that really 
can be quantified, and the horse representing other 
considerations, including environmental spillovers 
(410). “No matter how carefully the scientific rabbit 
is chosen, the flavor of the resulting stew is sure to be 
swamped by the horse flesh,” he concludes. Econo- 
mists should resist the temptation to ignore the horse. 


Implications of the Bias of Science 
Toward Rationality 


Both distributional considerations and the truth or 
falsity of the assigned values in CBA are value/ethi- 
cal problems with which many of the advocates of 
such analysis are all too familiar and with which they 
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are constantly struggling. Another value issue is 
more subtle and does not appear to have been previ- 
ously identifed, no doubt partly because it underlies 
not just CBA, nor even economics, but the whole 
ethos of the scientific enterprise. That is the bias of 
science toward knowledge as a good for its own sake. 
This bias values “rational” decisions (usually defined 
as decisions made on the basis of “fact’’) over “in- 
tuitive” ones (usually characterized by a “gut feel- 
ing’). 

As Charles Lindblom and David Cohen empha- 
size, policy analysts tend to believe that all problems 
are best solved by gaining more information (361). 
Over and over again, CBA is justified not on grounds 
that it is accurate or fair, but that it represents in- 
formation, and in that sense is and will always be 
preferred to the prior state of ignorance. Nobody 
puts the argument more succinctly than Prest and 
Turvey, who note that CBA forces those who con- 
duct it to quantify costs and benefits insofar as possi- 
ble (487): 

. . rather than rest content with vague qualitative 
judgments or personal hunches. This is obviously a 
good thing in itself; some information is always better 
than none. 

Mancur Olson attributes the quest for rationality 
not just to science in general, but to economics in 
particular (469): 

. . . Economic (more precisely microeconomic) the- 
ory is in a fundamental sense more nearly a theory of 
rational behavior than a theory of material goods. 

Boulding grants that the dollar “is a dangerously im- 
perfect measure of the quality of human life and hu- 
man values” (67). Nevertheless, he says (67): 

. it is a useful first approximation, and in these 
matters of evaluation of different choices it is extreme- 
ly useful to have a first approximation that we can 
then modify. Without this, indeed, all evaluation is 
random selection by wild hunches. 

Layard points out that all the writers in his edited 
volume assume that knowledge is better than ignor- 
ance (349). 

Fein, who acknowledges serious problems in accu- 
rate valuation, nevertheless concludes “these prob- 
lems leave us no worse off than we are in the absence 
of the evaluation effort” (194). In fact, he says, our 
awareness of the problems may leave us better off 
(194): 

. . . the cost-benefit analysis leads to a greater level 
of understanding of the deficiencies in our measuring 
techniques, of the vagueness of some of our goals. It 
does not make us ignorant but makes us aware of our 
ignorance. It forces us to question the “conventional 
wisdom”... 

Although making comparisons between people or 
diseases is distasteful, Fein argues, such comparisons 


are being made all the time anyway. It is preferable 
to make the comparisons explicitly so that we can be 
aware that we are valuing the lives of airplane pas- 
sengers, for example, more highly than those of coal 
miners. 

Indeed, several writers justify CBA by arguing it 
makes us face squarely the moral implications of 
valuing different people’s lives and preferences dif- 
ferently. Parish observes (in connection with the will- 
ingness-to-pay principle and the difficulty of valuing 
intangibles not sold in markets) that even to point 
out these difficulties is useful because it reveals an 
area of ignorance (475). 

Mishan views the economists doing CBA as having 
a much more active role than that of a simple techni- 
cian (410). To him, the cost-benefit analyst is a moral 
agent with an obligation to incorporate some of the 
deficiencies revealed by the analysis into policy for- 
mulation. Even when analysts cannot bring some im- 
portant but unquantifiable item into the calculus, 
they can at least clearly reveal an area of ignorance. 
They can also help valuation by providing a physical 
description of the spillovers and their significance 
and offering a guess or range of guesses about the 
value of the damage. Analysts can also provide con- 
tingency calculations and estimate the critical mag- 
nitude for the spillovers that will just offset a proj- 
ect’s benefits. They can even use questionnaires, says 
Mishan (410): 

Surveys based on the questionnaire method may be 
suspect for a number of reasons, but they are some- 
times better than guesswork, and assuredly better 
than no information at all. 

Again and again, in discussion after discussion, 
commentators state the better-than-nothing argu- 
ment. Despite these expressions of faith, the question 
remains whether revealing our ignorance will, in 
fact, force us to face and remedy it. Grosse observes 
that the HEW study of maternal and child health pro- 
grams was a difficult one to do because hard infor- 
mation on the state of children’s health was difficult 
to obtain (265): 

Surprisingly, estimates of improvement in general 
health attributable to medical care are almost nonex- 
istent. It is not easy to demonstrate statistically that 
children who see doctors regularly are healthier than 
children who do not. 

This statement is an indication of what we can prob- 
ably take to be Grosse’s belief that visits to the doctor 
do indeed improve health; he is frustrated only by his 
inability to demonstrate that statistically. The alter- 
native view, both plausible and increasingly fashion- 
able, is that visits to the doctor do not contribute ap- 
preciably to the general health of children. That 
possibility appears not to have occurred to Grosse, 
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though entertaining it might certainly give policy- 
makers quite a different view of what kinds of prior- 
ities to accord to health care. 

One of the most extensive criticisms of this ration- 
alism-oriented mind set has been lodged by Veatch. 
He argues that systems analysis, rationalized cen- 
tralized planning, and quantification are not value- 
neutral, but are instead attractive to people who have 
certain values (603). Veatch criticizes systematic, 
data-based analysis of policy problems on two 
grounds. One is practical— he has doubts about its 
outcome. So many things in practice turn out to be 
unquantifiable, Veatch says, that it may be better not 
to pursue such analysis at all because the possibility 
of error may be great. Furthermore, Veatch says, sys- 
tematic, data-based analyses of policy problems are 
contrary to a sense of human freedom or spontaneity 
and are overly rationalistic (603): 

It is sobering to realize that basic lifestyle choice is 
at stake here. One cannot systematically or rationally 
prove that systematic, rational policy analysis is a 
preferable style of living in a social community. 

For Lindblom and Cohen, the problem is not style 
but substance (361). They believe that more knowl- 
edge will be of little use in many situations and that 
many problems are best solved “interactively,” that 
is, by means of negotiation or the political process. 
Even when knowledge is of value, it is usually only 
one element in a more complex political and human 
situation. Moreover, they argue, interactive problem 
solving can be just as rational in its methodology as 
the “scientific” approach. 

Economics, like the rest of science, however, is tied 
irrevocably to the latter. As Boulding observes (67): 

The fundamental principle that we should count all 
costs, whether easily countable or not, and evaluate 
all rewards, however hard they are to evaluate, is one 
which emerges squarely out of economics and which 
is at least a preliminary guideline in the formation of 
the moral judgment, in what might be called the “eco- 
nomic ethic.” 

Boulding freely acknowledges that some things sim- 
ply cannot be measured, particularly what he calls 
the “heroic ethic” (bravery in the military, religious 
feelings, and so forth) (67): 

The attack on economics is an attack on calculated- 
ness, and the very fact that we think of calculating as 
cold suggests how exposed economists are to romantic 
and heroic criticism. 

The problem is to retain both heroic and economic 
elements in our institutions and in our decisionmak- 
ing, but in proper balance (67): 

Economic man is a clod, heroic man is a fool, but 
somewhere between the clod and the fool, human 
man, if the expression may be pardoned, steers his tot- 
tering way. 


Though most instinctively prefer even a flawed 
systematic analysis to none, it is well to keep Fein’s 
warning in mind (194): 

While it is better to know something than to know 
nothing . . . we dare not minimize the danger that in 
knowing something we may behave as if we know 
everything. 


implications of the Conflict Between 
Freedom and Paternalism 


The conflict between freedom and paternalism is 
one not much discussed in the literature on CBA. But 
individual freedom is an issue particularly important 
in the United States, and paternalism one particularly 
important in medicine. The conflict between freedom 
and paternalism is thus a perennial one in U.S. health 
care. Clearly, CBA, with its quantitative methods ~ 
and scientific respectability, can force choices. 

Nevertheless, it is anything but clear that CBA has 
done this. One of the early analyses performed at 
HEW and described by Grosse (265), for example, ex- 
amined the value of fluoridation of drinking water to 
help reduce cavities. Grosse says that on paper (265): 

. . . fluoridation looks like a very attractive pro- 

gram. It was so attractive that one can assume that a 

program as cheap as this is not bring inhibited by a 

lack of financial support by the Federal Government; 

there are other factors at work. 
He does not specify the nature of those other factors, 
but most of us know they are political. There is 
strong local opposition to fluoridation of water, 
often on grounds that it is simply wrong to force 
everybody in a community to undergo a medical 
treatment with no way of opting out. 

Public policy, particularly health policy, has often 
been criticized as paternalistic. The Government is 
often accused of meddling in people’s lives, trying to 
get them to behave in certain ways for their own 
good. CBAs are, in a way, a response to that criti- 
cism. Today, health programs are justified less on 
grounds that the Government ought to be doing good 
things for its citizens than in terms of the tax savings 
and other economic benefits that will result. Thus, 
screening newborns for PKU is no longer (or rarely) 
supported on grounds that families ought to be 
spared the miseries of having to cope with a retarded 
child who need not have been so retarded, but rather 
on grounds that the Government will save money on 
institutional costs for the retarded by underwriting 
such a program. 

It is probable, however, that in a way CBA func- 
tions partly as a mask for the old paternalism. The 
extensive analyses of genetic screening, for instance, 
offer a respectably scientific and economic justifica- 
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tion for continuing programs whose origins are actu- 
ally traceable to a combination of political and hu- 
mane impulses. This illustrates that the important 
problem here may not be the paternalism of econo- 
mists, but that of the policymaker. The Government 
policymaker orders and pays for the analysis. Be- 
cause of this, the policymaker will often have much 
to say about what goes into the analysis, for in- 
stance, concerning goals and outcomes, Further- 
more, the policymaker is also in a position to apply 
the analysis to the making of social decisions. This 
position not only invites paternalism, it almost 
demands it, since in some sense policymakers are by 
definition in a somewhat paternal role with respect to 
the people for whom they make policy. 

Of course, a CBA can also be influenced by the 
degree of paternalism subscribed to by economists. 
For example, many analyses employ shadow pricing 
in which, for a variety of reasons, the value of some 
item in the analysis is set at something other than its 
current market value. As Layard points out, highly 
paternalistic economists may very well substitute 
their own valuation for that revealed by answers to 
survey questions if they believe that the public's 
valuations may not be accurate—particularly when 
valuing future possibilities (349). 


Conclusions 


The issues of equity, efficiency, freedom, pater- 
nalism, the value of knowledge, and rationality—by 
truth-telling and deception—are irretrievably im- 
bedded in the techniques of CEA and CBA. There is 
no way to change the techniques to eliminate those 
values and their inevitable conflicts. Certain tech- 
nical adjustments (e.g., putting the services of house- 
wives into dollar terms) have been made in the past, 
and these have improved the situation somewhat. 
Similar adjustments will continue to be made in the 
future, but essentially they will be small adjustments 
that will have no effect on the fact that the issues de- 
scribed are simply inherent in the techniques them- 
selves or in their translation into policy. 

Should we not use cost-benefit techniques in forg- 
ing health policy? No, discarding CBA is not the 
answer. In an important sense, the defenders of the 
technique are correct when they argue that policy 
decisions in health care are being made daily on 
shaky grounds anyway. CBA is at least an attempt, 
however imperfect, to ground those decisions in real 
needs and real possibilities. 

The problem is not that CBA lacks objectivity and 
is not value free, but rather that objectivity and value 
freedom are unjustifiably attributed to it. To their 
credit, most theorists of CBA cited here are quite 


aware that their techniques are anything but objec- 
tive and value free. The problem comes with lesser 
practitioners who see themselves simply as techni- 
cians or—and this may be the real problem—with 
policymakers who take the scientific aura of CBA for 
truth, failing to realize CBA’s limitations. 

Many, discussions of CBA have paid little atten- 
tion to policymakers. Yet, virtually all analyses are 
undertaken at the behest of policymakers, and the 
methods employed in a specific analysis may depend 
on their goals and values. Weinstein and Stason 
point out, for instance, that a health maintenance 
organization may care about total costs, whereas a 
group practice may care chiefly about nonreimburs- 
able costs; such differences have obvious and impor- 
tant implications for society's efforts to contain costs 
(627). 

To understand the pros and cons of CBA studies, 
one must also understand the policy level on which 
they are commissioned and applied. Who is the pol- 
icymaker? The Secretary of DHHS, the Senate Sub- 
committee on Health and Scientific Research, the 
State health department, the health systems agency, 
the hospital administrator, the school nurse, the pri- 
vate physician? Each of these policymakers may very 
well use CBA to make policy. The decision may be 
different in every case, however, because individual 
policymakers, like economists, will have personal 
values, institutional biases, and a somewhat different 
constituency from others’. 

Harold Green notes that optimism or pessimism 
can sometimes be “a function of statutory mandate 
or agency mission” (258). The Food and Drug Ad- 
ministration, for instance, automatically regards any 
food additive as hazardous until it has been demon- 
strated to be safe. The importance of the policy- 
maker's perspective is illustrated, as well, by the 
unsettled question of whether a physician ought to 
depart from concern for the patient’s welfare and 
begin to incorporate social values into the decision 
to, for example, order an additional test that has only 
very small chance of giving additional information. 

For some analysts, the question of who the policy- 
maker is does not seem to be very important. If as an 
analyst you follow Mishan’s recommendations (410), 
for instance, you will behave quite explicitly as a 
moral agent, with a duty to point up all the relevant 
effects of a program that do not enter the cost-benefit 
computation no matter who your employer is. Mi- 
shan also argues that if economists have reason to be- 
lieve that a project will be unambiguously regressive, 
it is their duty to mention this; for Mishan, econo- 
mists should be just as committed to equity as to effi- 
ciency. That commitment should obtain, he implies, 
regardless of the level on which policy is being made. 
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It is not clear whether Mishan’s view is widely 
shared by economists, or whether instead most econ- 
omists see themselves as simple hired hands. At a 
minimum, though, it seems sensible to assert that 
there are some kinds of information that ought to be 
made explicit as part of a CBA. These include clear 
indications of the data sources on valuing lives, what 
kinds of equity considerations spring from the use of 
particular data (e.g., whether whites are valued more 
highly than blacks), and how and why the discount 
rate was selected. 

Conventional assumptions, disclaimers, and cau- 
tions also ought to be an explicit part of each CBA. 
Greater use of sensitivity analysis would probably be 
illuminating. In addition, people need to be made 
more aware of the limitations of CBA. That probably 
includes people within the economics profession 
itself, certainly includes policymakers who use CBA, 
and may even include the general public. 

CBA should certainly never be used as the sole ba- 
sis for major health policy decision. Such use, how- 
ever, may not be as much of a problem as has often 
been charged. Grosse for instance, says the HEW 
analyses were used to give the Department a “feel for 
what were relatively high- and low-priority pro- 
grams, and then to feed these insights into the deci- 
sionmaking process, which also considers other 
viewpoints, the existing commitments, the political 
situation, the rate of spending, and the ability to get 
people moving on programs, and so on” (265). In 
Grosse’s view, CBA is clearly a useful tool, but only 
one of many. 

Swint and Nelson apparently share this view, ob- 
serving (593): 

Many of the criticisms of (cost-benefit analysis) 
come from the people who have (perhaps inadvertent- 
ly) set up a “straw man” by claiming it cannot provide 
the sole decision-making criterion. The point is that 
(cost-benefit analysis) is not intended to provide the 
sole decision-making criterion; that has only been 
incorrectly imputed to it by non-practitioners. What it 
does do is provide economic information that must be 
combined with distributional, sociopolitical, humani- 
tarian and other information by the decision maker 
(vis-a-vis the analyst) for net evaluation. We see no 
point in depriving decisionmakers of one piece (the 
economic) of the several sets of information needed to 
make rational decisions. 

For Prest and Turvey, who were writing, after all, 
in the comparatively early days of 1965, CBA is 
“only a technique for making decisions within a 
framework which has to be decided upon in advance 
and which involves a wide range of considerations, 
many of them of a political or social character” (487). 
They argue that the case for using CBA is strength- 
ened, not weakened, if its limitations are openly 
recognized and emphasized. CBA may then function 


as a “sensible antidote to the wilder excesses of par- 
ticular lobbies.” CBA may also cause questions to be 
asked that would not otherwise have been raised. 
Prest and Turvey also agree with Grosse that even if 
CBA cannot always give the right answers, it can 
sometimes play the purely negative role of screening 
projects and rejecting those that are clearly less 
promising (487,265). 

On the other hand, one should certainly not under- 
estimate the power of politics. Klarman points out 
that politics is one of the foremost barriers to system- 
atic analysis (265). CBA implies a delineation of 
goals and an articulation of values, whereas the pollit- 
ical process, he says, may require the blurring of dif- 
ferences and conflicts in order to facilitate coalition 
building so that particular ends can be achieved. Ac- 
cording to Martin Rein, because of that requirement 
of politics, it is necessary to put planning beyond the 
reach of shortsighted political considerations: 
“... at some stage, and to some degree, decisions 
must be taken out of the political context and handed 
over to the social scientist for analysis” (501). Real 
exploration and analysis of alternative choices, he 
suggests, can occur only in the absence of short-term 
political pressures. 

How decisions can be taken out of the political 
context for analysis is not clear. Nor is it entirely 
clear that they should be. Green argues that some- 
times nonscientific negotiation and compromise may 
be the best course for policymakers (258,361). He 
asks (258): 

Is it not more important to resolve controversy in 
the optimum manner than to produce an objectively 
correct result? If so, is it not possible that optimum 
resolution of controversy may require acceptance as 
true of something that we know is untrue? 

Even some of CBA’s best-known and most sensi- 
tive practitioners acknowledge that, at times, the 
political process may respond better to our needs 
than even the most rational and careful CBA. Rashi 
Fein believes in CBA. He thinks that policymakers 
need to be reminded that there are economic returns 
to health programs, that good health can be sup- 
ported on investment grounds, and that poor health 
costs a lot. On the other hand, Fein comments (194): 

At present, in the United States, beset as we are by 
divisions and by tensions, distributional considera- 
tions lie at the heart of many of our problems. The 
healing of social wounds (not an unimportant objec- 
tive even if its benefits cannot be quantified in mone- 
tary returns) may, today, be more readily accom- 
plished by providing the services that people believe 
to be important than by providing that which the ana- 
lyst has tentatively determined is most beneficial. The 
healing of social wounds, is, at this moment, I believe 
more vital than the healing of disease. 


Appendix E.—Legal Status of CEA/CBA 





Most of the existing statutory authorities that 
guide Federal health-care-related agencies allow the 
agencies to use or support CEA/CBA. Recently, 
however, two legislative initiatives—Public Law 
95-623 establishing the National Center for Health 
Care Technology (NCHCT) and the National Health 
Planning and Resources Development Amendments 
to Public Law 93-641—placed special emphasis on 
the use and support of CEA by NCHCT and by 
health systems agencies (HSAs). Although these laws 
are not mandates, they are explicit suggestions to do 
CEA or CBA studies. The language of Public Law 
95-623 calls for NCHCT to “give appropriate empha- 
sis” to cost effectiveness (among a number of other 
criteria) while conducting and supporting research on 
health care technology. Similarly, the health plan- 
ning amendments state that cost effectiveness should 
be one of the criteria that HSAs use while carrying 
out their mandate to review the appropriateness of 
existing facilities (see ch. 7 for a detailed discussion). 
These two laws are as close as Congress has come to 
requiring a health-care-related agency to perform or 
support CEA or CBA studies as part of its mission. 

At the other end of the legislative spectrum from 
these two laws are the statutes that authorize and 
guide the efforts of the Food and Drug Administra- 
tion (FDA). FDA has traditionally interpreted its 
mandate as excluding or strongly discouraging the 
formal evaluation and incorporation of economic 
costs and social benefits in its regulatory proceedings 
(127a). Although, FDA has done a small number of 
costs and benefits type comparisons of proposed in- 
itiatives, those analyses were conducted in response 
to executive orders calling for: Inflationary Impact 
Statements (E.O. 11821, 1974), Economic Impact 
Statements (E.O. 11949, 1977), and Improving Gov- 
ernment Regulations (E.O. 12044, 1978). Those 
orders applied to executive agencies that proposed 
major regulations, i.e., regulations having an annual 
impact on the economy of $100 million or more or 
causing a major increase in costs or prices for individ- 
ual industries, levels of government, or geographic 
regions. Any agency that proposed a regulation fall- 
ing into one of these categories was required to ex- 
amine the costs and benefits of the proposed action. 

One section of the food additive amendments to 
FDA's mandate, usually referred to as the “Delaney 
clause,” (The Food, Drug, and Cosmetic Act, 21 
U.S.C., sec. 409c3a, 1976) explicitly prohibits the 
balancing of risks and benefits in a proposed regula- 
tion of a food additive that has been shown to be car- 
cinogenic (127a). The inflexible language of this 
amendment has stimulated a number of legislative 


proposals to repeal or alter the “Delaney clause,” 
and/or alter FDA's authorizing legislation to allow or 
require the balancing of risks and benefits in pro- 
posed regulatory or rulemaking initiatives.1 The 
most prominent of the initiatives is the bill intro- 
duced in the 96th Congress as the “Drug Regulation 
Reform Act of 1979.” 

The legal status of CEA and CBA in the decision- 
making processes of other agencies of the Federal 
Government is in a much greater state of conflict, 
confusion, and flux. In the past 10 years, numerous 
court rulings have attempted to define and clarify the 
role of CEA and CBA in the decisionmaking proc- 
esses of regulatory agencies. Increasing congressional 
attention is being focused on the potential usefulness 
of CEA and CBA techniques, especially as they per- 
tain to the decisionmaking efforts of several regula- 
tory agencies. Over 65 bills were introduced in the 
last two Congresses that would require various agen- 
cies to incorporate CEA, CBA, or risk-benefit tech- 
niques into their formal decisionmaking procedures 
(600). For many agencies, the legislation would sim- 
ply formalize a CEA/CBA process that is now infor- 
mal. In a number of other agencies, a formal or infor- 
mal process of using CEA/CBA does not exist. In any 
case, many of the proposed changes would make ex- 
plicit that which past legislation has left vague and 
open to judicial and agency interpretation. 

In the past decade, a number of health, safety, and 
environmental laws have added to the Federal Gov- 
ernment’s involvement in the regulation of highly 
controversial areas of society and business.” In many 
instances, the specific purpose and intended out- 
comes of the legislation were inadequately defined 
after the general goals were established. In addition, 
the types of procedures allowed or required to initi- 
ate agency action, or the level of “evidence” required 
to support a given initiative, were left vague in the 
authorizing legislation. In many cases, these things 
were left vague with good reason. The uncertainties 


‘Most of the reaction was stimulated by the use of the “Delaney clause” to 
initiate a ban of saccharin, an artificial sweetener used extensively in this 
country. For a detailed analysis of the technical issues involved in the pro- 
posed saccharin regulation, see Cancer Testing Technology and Saccharin 
(466), prepared by OTA. 

Included among these laws are (127a): 1) Safety and Health Act (Public 
Law 91-596); 2) Consumer Product Safety Act (Public Law 92-573); 3) 
Amendments to the Pesticide Act (Public Law 92-516, Public Law 94-140, 
Public Law 95-396); 4) Solid Waste Disposal Act (Public Law 94-580); 5) 
Toxic Control Act (Public Law 94-469); 6) Medical Device Amendments 
(Public Law 94-295); 7) Safe Drinking Water Act of 1977 (Public Law 
95-190); 8) Clean Air Act Amendments of 197? (Public Law 91-604, Public 
Law 95-95). Perhaps the most well known is the National Environmental 
Policy Act of 1969— NEPA (U.S.C. sec. 4321 et seq., 1970). It has been re- 
sponsible for a significant number of legal challenges regarding the deci- 
sionmaking methods of a host of Federal agencies. 
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involved in the issues, the imprecise nature of the 
best available data, and many other factors required 
that the authorizing statutes remain flexible. 

The broadly worded statutory language, however, 
has left the door open for procedural challenges, dif- 
fering interpretations of the laws, and litigation. The 
practical result of this problem is that the courts have 
been repeatedly forced to clarify and define what is 
required of the agencies in establishing a rule or reg- 
ulation. Numerous court cases have challenged the 
decisionmaking procedures of the Environmental 
Protection Agency, the Consumer Product Safety 
Commission (CPSC), the Nuclear Regulatory Com- 
mission, the Occupational Safety and Health Admin- 
istration (OSHA), and others.? One of the results of 
this litigation is an increasing pressure on the agen- 
cies to include more formal and explicit costs v. bene- 
fits comparisons in their decisionmaking and evi- 
dence-gathering procedures. 

Neither the courts nor Congress are totally unfa- 
miliar with CEA or CBA techniques. The Federal 
courts have been concerned with agency actions 
using costs and benefits since the early 1940's (526). 
Congress has required the Corps of Engineers to jus- 
tify water resource projects with cost-benefit calcula- 
tions since the late 1930’s. Recent years, especially 
the last 10, however, have witnessed an increasing 
awareness of the use of CEA/CBA in the regulatory 
decisionmaking process. Enactment of the National 
Environmental Policy Act (NEPA) in 1969 and the 
issuance of the executive orders mentioned above 
marked the beginning of significant judicial involve- 
ment in the area of procedural and substantive re- 
view of CEA/CBA use in the regulatory process. 

Despite a decade of litigation, conflict, and debate, 
the role of formal CEA/CBA in the regulatory proc- 
ess still remains ill-defined and unclear in many, if 
not most, decisionmaking situations. The courts 
have had, and continue to have, a difficult time in 
producing a coherent body of law that establishes 
uniform standards of judicial review in this area (this 
is especially true for NEPA-related issues). They have 
not been able to reconcile the conflicting standards 
that have been placed into various statutes that in- 
directly require many agencies to balance the social 
costs and benefits of their regulatory initiatives (526). 
The courts review agency initiatives with three basic 
criteria in mind: 1) Did the agency conform to the 
procedures set out in the law? (the procedural test); 
2) Was the cost-benefit comparison arbitrary or ca- 

’This appendix does not attempt to provide a comprehensive discussion 
of the development and interpretation of the case law that guides this‘area 
of regulatory behavior. At best, an attempt is made to touch on the recent 
judicial highlights that have built upon earlier rulings. For a more complete 


analysis of the courts’ involvement in this area, see references 41, 42, and 
526. 


pricious? (a substantive and due process test); and 3) 
What is the “substantial evidence on the record as a 
whole?” (also a substantive test). Putting these cri- 
teria into practice has not always been easy or suc- 
cessful. The limitations of the methods of CEA/CBA, 
the vagueness of the legislative language and intent, 
the differing interpretations of the level of sophistica- 
tion or rigor required in the cost-benefit compari- 
sons, and the reluctance of the courts to substitute 
their judgments for agency expertise have contrib- 
uted to the continuing confusion that surrounds the 
use of formal CEA/CBA techniques in the decision- 
making process. 

A number of recent and pending Federal District 
Court rulings may be able to give further indications 
of the direction in which the courts are moving with 
respect to requiring CEA or CBA techniques in the 
decisionmaking process. One should keep in mind, 
however, that the courts’ opinions may be specific to 
an agency, may be subject to later reinterpretation by 
a higher court, or may be applicable only to the spe- 
cific statutory wording on which the rulings turn. 
OTA is not suggesting that these opinions will have 
significant impact on the health-related agencies ex- 
amined in this assessment. As judicial signals that 
may very well broadcast to a number of executive 
agencies the extent to which formal CEA/CBA tech- 
niques might be used or required in their decision- 
making procedures, however, these opinions are 
worthy of note. 

The litigation many observers may be watching 
most closely is the “benzene case” now before the 
Supreme Court.‘ That case involves a number of is- 
sues, but the primary focus is on two questions: 1) Is 
OSHA required to use a costs v. benefits comparison 
to support its proposed standards? and, 2) If so, is the 
analysis OSHA claims to have performed adequate 
to support its decision? A lower court answered yes 
to the first and no to the second, and overturned 
OSHA’‘s proposed benzene standards for the work- 
place (20). The lower court held, in part, that OSHA 
had failed to properly or adequately compare the 
benefits expected from the proposed regulation to the 
anticipated costs of compliance to determine if a 
“reasonable relationship” existed.° 


‘Industrial Union Department, AFL-CIO v. American Petroleum, et al. 
(No. 78-911) and Marshall v. American Petroleum Institute, et al. (No. 
78-1036). 

SThe language in OSHA's mandate (Occupational Safety and Health Act, 
29 U.S.C. sec. 652 et seq. (1975)) requires that OSHA has the obligation to 
enact only standards that are “reasonably necessary” or appropriate to pro- 
vide safe or healthful workplaces and that standards dealing with toxic ma- 
terials be “feasible.” OSHA must determine whether benefits expected from 
a standard bear a reasonable relationship to costs imposed by the standard. 
The Circuit Court rulings in the CPSC and OSHA cases hinge on these 
phrases; “reasonably necessary,” “feasible,” and whether benefits expected 
bear a “reasonable relationship” to the costs expected. 


Circuit Court rulings involving CPSC were cited 
by the court in the benzene case to illustrate the evi- 
dence requirements that the court felt are necessary 
to determine if a proposed initiative is “reasonably 
necessary” when the proposed standard’s benefits are 
balanced against the costs, or burden, of compliance 
with the standard.* The Supreme Court will deter- 


*The rulings cited in the OSHA opinion as guiding the determinations 
were: 


1. Aqua Slide ‘N’ Dive v. Consumer Product Safety Commission, 569 F. 2d. 
832 (Sth Cir. 1978). and 

2. D. D. Bean & Sons v. Consumer Product Safety Commission. 574 F. 2d 
643 (1st Cir. 1978). 
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mine if OSHA’s statutory language requires it to go 
beyond the data it has used in the past and follow the 
guidelines established by the CPSC rulings for the use 
of costs v. benefits comparisons in its decisionmaking 
procedures. 

The available evidence strongly indicates that 
there is mounting legislative and judicial pressure to 
formalize the use of CEA/CBA in many decision- 
making areas in the Federal Government. To date, 
most of this pressure has been focused on the regula- 
tory agencies. It is uncertain at this time if these pres- 
sures will expand into the health care system. 


Appendix F.— Description of 


Other Volumes of the Assessment 


In addition to the present report, several back- 
ground papers were prepared in conjunction with 
OTA’s assessment The Implications of Cost-Effec- 
tiveness Analysis of Medical Technology, and these 
are available separately. The findings and options of 
the assessment are in large part derived from the in- 
formation generated by the background efforts. 


Background Paper #1: Methodological Issues and 
Literature Review, includes an in-depth examination 
of the decisionmaking context and methodology dis- 
cussions presented in this report. A critique of the lit- 
erature, a bibliography of over 600 items, and ab- 
stracts of over 70 studies and other articles are also 
included. 


In order to investigate the applicability of tech- 
niques to assess the costs and benefits of medical 
technology, 19 case studies were prepared. All 19 are 
available individually. In addition, 17 of the cases are 
available in a collected volume, Background Paper 
#2: Case Studies of Medical Technologies. Some of 
the cases represent formal cost-effectiveness analysis 
(e.g., the case on bone marrow transplants), while 
others represent net cost or “least cost” analysis (e.g., 
the case on certain respiratory therapies). Other cases 
illustrate various issues such as the difficulty of con- 
ducting cost-effectiveness analysis in the absence of 
adequate efficacy and safety information (e.g., the 
case on breast cancer surgery), or the role and impact 
of formal analysis on policymaking (e.g., the case on 
end-stage renal disease interventions). The 17 case 
studies in Background Paper #2 and their authors 
are: 

Artificial Heart 

Deborah P. Lubeck 

John P. Bunker 
Automated Multichannel Chemistry Analyzers 

Milton C. Weinstein 

Laurie A. Pearlman 
Bone Marrow Transplants 

Stuart O. Schweitzer 

© CC. Sealz 
Breast Cancer Surgery 

Karen Schachter 

Duncan Neuhauser 
Cardiac Radionuclide Imaging 

William B. Stason 

Eric Fortess 
Cervical Cancer Screening 

Bryan R. Luce 
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Cimetidine and Peptic Ulcer Disease 

Harvey V. Fineberg 

Laurie A. Pearlman 
Colon Cancer Screening 

David M. Eddy 
CT Scanning 

Judith L. Wagner 
Elective Hysterectomy 

Carol Korenbrot 

Ann B. Flood 

Michael Higgins 

Noralou Roos 

John P. Bunker 
End-Stage Renal Disease Interventions 

Richard A. Rettig 
Gastrointestinal Endoscopy 

Jonathan A. Showstack 

Steven A. Schroeder 
Neonatal Intensive Care 

Peter Budetti 

Peggy McManus 

Nancy Barrand 

Lu Ann Heinen 
Nurse Practitioners 

Lauren LeRoy 

Sharon Solkowitz 
Orthopedic Joint Prosthetic Implants 

Judith D. Bentkover 

Philip G. Drew 
Periodontal Disease Interventions 

Richard M. Scheffler 

Sheldon Rovin 
Selected Respiratory Therapies 

Richard M. Scheffler 

Morgan Delaney 

The 18th case study is published separately as 
Background Paper #3: The Efficacy and Cost-Effec- 
tiveness of Psychotherapy. That study assesses meth- 
odological and substantive issues relating to the 
scope of psychotherapy, the evaluation of psycho- 
therapeutic efficacy, and the applicability of cost-ef- 
fectiveness and cost-benefit analysis in assessing psy- 
chotherapy. It was prepared by Leonard Saxe on the 
basis of a report prepared for OTA by Brian Yates 
and Frederick Newman. The 19th case study was pre- 
pared for OTA by Judith Wagner and is published 
separately as Background Paper #5: Assessment of 
Four Common X-Ray Procedures. 

Background Paper #4: The Management of Health 
Care Technology in Ten Countries is an analysis of 


the policies, programs, and methods, including cost- 
effectiveness and cost-benefit techniques, that nine 
industrialized nations other than the United States 
use to manage the effects of medical technology. The 
experience of these nine countries in managing medi- 
cal technology is compared to that of the United 
States. The paper on the United States and the com- 
parative analysis were prepared by OTA staff, as- 
sisted by Louise Russell. The authors of the papers on 
the nine foreign countries are: 
United Kingdom 

Barbara Stocking 
Canada 

Jack Needleman 
Australia 

Sydney Sax 
Japan 

Joel Broida 
France 

Rebecca Fuhrer 
Germany 

Karin A. Dumbaugh 
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Netherlands 

L. M. J. Groot 
Iceland 

David Gunnarson 

Duncan vB. Neuhauser 
Sweden 

Erik H. G. Gaensler 

Egon Jonsson 

Duncan vB. Neuhauser 

A related report prepared by OTA and reviewed 
by the Advisory Panel to the overall assessment is A 
Review of Selected Federal Vaccine and Immuniza- 
tion Policies. That study, published in September of 
1979, examined vaccine research, development, and 
production; vaccine efficacy, safety, and cost-effec- 
tiveness; liability issues; and factors affecting the use 
of vaccines. Pneumococcal vaccine was used as a 
case study, and a cost-effectiveness analysis was per- 
formed. 
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ACRONYMS 


ADAMHA Alcohol, Drug Abuse, and Mental 


AHA 
AIP 
AMA 
ANDA 
AoA 
ASH 
ASPE 


BC/BS 
BCHS 


BHF 
BHP 
BHPr 
BMD 
BMS 


BOB 
BOD 
BOE 
BOL 
BRH 
CBA 
CBO 


CDC 
CEA 
CEA/CBA 


CFR 
CHP 
CMHC 
CON 
CroG 
Ch 
DARP 
DES 
DESI 


DHEW 
DHHS 


DOD 
DOE 
DOL 
DRG 
EFM 


Health Administration (PHS) 

American Hospital Association 

annual implementation plan 

American Medical Association 

abbreviated new drug application 

Administration on Aging 

Assistant Secretary for Health (DHHS) 

Assistant Secretary for Planning and 
Evaluation (DHHS) 

Blue Cross and Blue Shield Associations 

Bureau of Community Health Services 
(Health Services Administration) 

Bureau of Health Facilities (HRA) 

Bureau of Health Planning (HRA) 

Bureau of Health Professions (HRA) 

Bureau of Medical Devices (FDA) 

Bureau of Medical Services (Health 
Services Administration) 

Bureau of Biologics (FDA) 

Bureau of Drugs (FDA) 

Bureau of Epidemiology (CDC) 

Bureau of Laboratories (CDC) 

Bureau of Radiological Health (FDA) 

cost-benefit analysis 

Congressional Budget Office (U.S. 
Congress) 

Center for Disease Control (PHS) 

cost-effectiveness analysis 

cost-effectiveness analysis /cost-benefit 
analysis (when referred to as a class of 
analytical techniques) 

Code of Federal Regulations 

Comprehensive Health Planning 

community mental health center 

certificate of need 

Consumer Product Safety Commission 

computed tomography (scanner) 

Drug Abuse Reporting Program (NIDA) 

diethylstilbestrol 

Drug Efficacy Study Implementation 
(project) 

Department of Health, Education, and 
Welfare (now DHHS) 

Department of Health and Human 
Services (formerly DHEW) 

Department of Defense 

Department of Energy 

Deparment of Labor 

Division of Research Grants (NIH) 

electronic fetal monitoring - 


EPA 
EPSDT 


ESRD 
FDA 
FR 
Ey 
GAO 


GHAA 
GHC 


HCFA 
HEW 


HMO 
HRA 
HSA 
HSI 
HSr 
HSQB 


IDE 
IHS 


IND 


IOM 
IPA 
KFMCP 
MCE 


NAS 
NASA 


NCHOT 
NCHS 
NCHSR 
NCI 
NDA 
NEI 
NEPA 
NHLBI 


NIA 
NIAAA 


Environmental Protection Agency 

Early and Periodic Screening, Diagnosis, 
and Treatment (Program) (HEW) 

end-stage renal disease 

Food and Drug Administration (PHS) 

Federal Register 

fiscal year 

General Accounting Office (U.S. 
Congress) 

Group Health Association of America 

Group Health Cooperative; Seattle, 
Wash. 

Health Care Financing Administration 
(DHHS) 

Department of Health, Education, and 
Welfare (now DHHS) 

health maintenance organization 

Health Resources Administration (PHS) 

health systems agency 

health status index 

health systems plan 

Health Standards and Quality Bureau 
(HCFA) 

investigational device exemption 

Indian Health Service (Health Services 
Administration) 

notice of claimed investigational 
exemption for a new drug 

Institute of Medicine (NAS) 

individual practice association (HMO) 

Kaiser Foundation Medical Care Plan 

medical care evaluation studies (by 
PSROs) 

National Academy of Sciences 

National Aeronautics and Space 
Administration 

National Center for Health Care 
Technology (OASH) 

National Center for Health Statistics 
(OASH) 

National Center for Health Services 
Research (OASH) 

National Cancer Institute (NIH) 

new drug application 

National Eye Institute (NIH) 

National Environmental Policy Act 

National Heart, Lung, and Blood 
Institute (NIH) 

National Institute on Aging (NIH) 

National Institute on Alcohol Abuse and 
Alcoholism (ADAMHA) 


NIAID National Institute of Allergy and 
Infectious Diseases (NIH) 
NIAMDD National Institute of Arthritis, 
Metabolism, and Digestive Diseases 
(NIH) 


NICHD _ National Institute of Child Health and 
Human Development (NIH) 

NIDA National Institute on Drug Abuse 
(ADAMHA) 

NIDR National Institute of Dental Research 
(NIH) 

NIEHS National Institute of Enviromental 
Health Sciences (NIH) 

NIGMS _ National Institute of General Medical 
Sciences (NIH) 

NIH National Institutes of Health (PHS) 

NIMH National Institute of Mental Health 
(ADAMHA) 

NINCDS _ National Institute of Neurological and 
Communicative Disorders and Stroke 
(NIH) 

NIOSH _ National Institute for Occupational 
Safety and Health (CDC) 

NLM National Library of Medicine (NIH) 

NSF National Science Foundation 

OASH Office of the Assistant Secretary for 
Health (DHHS) 

OASPE Office of the Assistant Secretary for 
Planning and Education (DHHS) 

ODPHP Office of Disease Prevention and Health 
Promotion (OASH) 

OHDS Office of Human Development Services 

(DHHS) 

OHMO _—Offfice of Health Maintenance 
Organizations (OASH) 

OHRST  Offfice of Health Research, Statistics, and 
Technology (OASH) 


GLOSSARY 


Ambulatory medical care: Medical goods and serv- 
ices rendered to a patient not admitted to a hospi- 
tal or other inpatient health care facility, including 
such items as physician office visits, outpatient 
laboratory diagnostic services, and outpatient pre- 
scription drugs. 

Biomedical and behavioral research: A combination 
of biological, medical, psychological, social, and 
physical scientific investigations focused on eradi- 
cating disease and generating new scientific knowl- 
edge. 

Capitation financing method: The method of paying 
for medical care on a fixed, periodic prepayment 
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OMAR Office of Medical Applications of 
Research (NIH) 

OPE Office of Planning and Evaluation 
(OASH) 

OPEL Office of Planning, Evaluation, and 
Legislation 

OPPR Office of Policy Planning and Research 
(HCFA) 

OSHA Occupational Safety and Health 
Administration (DOL) 

OTA Office of Technology Assessment (U.S. 
Congress) 

OTC over-the-counter (Drug Review 
Program) (FDA) 

PDP product development protocol 

PGP prepaid group practice (HMO) 

PHS Public Health Service (DHHS) 

PPB planning-programing-budgeting 

PMA Pharmaceutical Manufacturers’ 
Association 

PSRO Professional Standards Review 
Organization, or Office of PSRO 
(HCFA) 

QALY quality-adjusted life year 

RFP request for proposal 

RMP Regional Medical Program 

SHCC State Health Coordinating Council 

SHPDA _ State health planning and development 
agency 

SSA Social Security Administration (DHHS) 

TOPS Treatment Outcome Prospective Study 
(NIDA) 

VA Veterans Administration 

ZBB zero based budgeting 


basis per individual enrolled in a health plan. Pay- 
ment by “capitation” implies that the amount paid 
by the individual is independent of the number of 
services that individual has received. 

CEA/CBA: A composite term referring to a family of 
analytical techniques that are employed to com- 
pare costs and benefits of programs or technol- 
ogies. Literally, the term as used in this assessment 
means “cost-effectiveness analysis/cost-benefit 
analysis.” 

Certificate of need (CON): A regulatory planning 
mechanism required by the National Health Plan- 
ning and Resources Development Act to control 
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large health care capital expenditures. CON appli- 
cations by institutions are reviewed by local health 
systems agencies, who recommend approval or 
disapproval; they are denied or approved by State 
health planning agencies (SHPDAs). 

Consensus development conference: A process in 
which knowledgeable biomedical researchers, 
practicing health professionals, and others, as ap- 
propriate, are brought together by NIH to explore 
publicly the scientific background, state of knowl- 
edge, proper use(s), and any other issues pertinent 
to the technology under consideration. 

Controlled clinical trial: An experimental design by 
which human or animal subjects are assigned, in 
accordance with predetermined rules, either to an 
experimental group (in which subjects receive a 
clinical intervention or dosage level of uncertain 
efficacy or safety) or to a control group (in which 
subjects receive some other intervention or dosage 
level, usually the customary or conventional one, 
or a placebo). If the predetermined rules specify 
that the subjects are assigned to groups randomly, 
the result is a randomized controlled clinical trial. 
The vast majority of randomized clinical trials are 
also controlled trials. 

Cost-benefit analysis: An analytical technique that 
compares the costs of a project or technological ap- 
plication to the resultant benefits, with both costs 
and benefits expressed by the same measure. This 
measure is nearly always monetary. 

Cost-effectiveness analysis: An analytical technique 
that compares the costs of a project or of alter- 
native projects to the resultant benefits, with cost 
and _ benefits/effectiveness not expressed by the 
same measure. Costs are usually expressed in dol- 
lars, but benefits/effectiveness are ordinarily ex- 
pressed in terms such as “lives saved,” “disability 
avoided,” ‘‘quality-adjusted life years (QALYs) 
saved,” or any other relevant objectives. Also, 
when benefits/effectiveness are difficult to express 
in a common metric, they may be presented as an 
“array.” 

Device (medical): Any physical item, excluding 
drugs, used in medical care (including instruments, 
apparatus, machines, implants, and reagents). 

Discount rate: A factor used in economic analysis to 
reduce to present value those costs and effects that 
occur in future years. Discounting is based on two 
premises: 1) individuals prefer to receive benefits 
today rather than in the future, and 2) resources in- 
vested today in alternative programs could earn a 
return over time. 

Drug: Any chemical or biological substance that may 
be applied to, ingested by, or injected into humans 


in order to prevent, treat, or diagnose disease or 
other medical conditions. 

Effectiveness: Same as efficacy (see below) except 
that it refers to’. . . average conditions of use.” 
Efficacy: The probability of benefit to individuals in 
a defined population from a medical technology 
applied for a given medical problem under ideal 

conditions of use. 

Fee-for-service: A method of paying for medical care 
on a retrospective basis by which each service ac- 
tually received by an individual bears a related 
charge. 

Health maintenance organization: A health care 
organization that acts as both insuror and provider 
of comprehensive medical services by a defined set 
of physicians to a voluntarily enrolled population 
paying a prospective per capita fee (i.e., paying by 
“capitation”). 

Health status index (HSI): A composite measure of 
well-being of an individual or a population. An 
HSI may include any two or more measures such 
as morbidity, mortality, level of function or mobil- 
ity, and potential for developing disease. 

Health services research: A field of inquiry that 
focuses on the structure, production, distribution, 
and effects of delivering personal health services. 

Health systems agency (HSA): One of the local 
health planning agencies which are charged with 
developing local health planning goals and imple- 
menting plans in consonance with State and Na- 
tional health care goals. HSAs are federally funded 
and are governed by a body which is broadly rep- 
resentative of both provider and consumer inter- 
ests, the latter being in the majority. The country is 
divided into areas each of which is served by an 
HSA. 

Human capital: An economic concept used to assess 
“livelihood,” or the earnings potential of an in- 
dividual. It has often been used as a proxy for the 
value of life in terms of an individual’s productive 
capacity to society. 

Investigational new drug (IND) application: Short 
for “notice of claimed investigational exemption 
for a new drug.” An IND application is submitted 
to FDA by a drug’s sponsor. It is a request for per- 
mission to use the drug in humans in order to in- 
vestigate the drug’s safety and efficacy for specific 
conditions. It must include the results of toxicity 
studies in animals, the qualifications of the investi- 
gators, and the design of the proposed clinical 
studies. 

Marginal benefit: An economic concept referring to 
the additional benefit achieved by incurring an ad- 
ditional unit of cost. 


Marginal cost: An economic concept referring to the 
additional cost of achieving one more unit of bene- 
fit. 

Medical care evaluation studies: Intensive retrospec- 
tive reviews of patients’ records to evaluate the 
quality, variation, and utilization of health care 
services delivered to the patient. 

Medical technology: The drugs, devices, and medical 
and surgical procedures used in medical care, and 
the organizational and support systems within 
which such care is provided. 

Morbidity: Illness, injury, impairment, or disability 
in an individual. 

Mortality: The death of an individual; often used in 
epidemiological studies where mortality rates for a 
population for a certain disease or injury are calcu- 
lated. 

Net cost analysis: A form of CEA/CBA (see above) 
that concentrates on costs, with less attention paid 
to analyzing outcomes in terms of health benefit. 
When alternatives are under study, their efficacy is 
often assumed to be equal. 

New drug application (NDA): An application to 
FDA by the sponsor of a new drug for permission 
to market the drug. The NDA must provide infor- 
mation that demonstrates the safety and efficacy of 
the drug. 

Procedure: A medical technology involving any 
combination of drugs, devices, and provider skills 
and abilities. Appendectomy, for example, may in- 
volve at least drugs (for anesthesia), monitoring 
devices, surgical devices, and physicians’, nurses’, 
and support staffs’ skilled actions. 

Professional Standards Review Organization: Com- 
munity-based nonprofit agencies directed by 
physicians that monitor the quality and appropri- 
ateness of institutional health care provided to 
medicare and medicaid beneficiaries. 

Quality-adjusted life year (QALY): A health status 
index in which 1 year of life is adjusted for various 
types and degrees of disability to yield 1 year of 
healthy life. QALYs are sometimes used to meas- 
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ure in common terms the effects on morbidity and 
mortality of health care technologies or programs. 

Risk: A measure of the probability of an adverse or 
untoward outcome’s occurring and the severity of 
the resultant harm to health of individuals in a de- 
fined population associated with use of a medical 
technology applied for a given medical problem 
under specified conditions of use. 

Risk-benefit analysis: The formal comparison of the 
probability and level of adverse or untoward out- 
comes versus positive outcomes for any given ac- 
tion. The comparison of outcomes does not take 
into consideration the resource costs involved in 
the intended action. 

Safety: A judgment of the acceptability of risk (see 
above) in a specified situation. 

State Health Coordinating Council (SHCC): The 
State advisory body to the State health planning 
and development agency (SHPDA). SHCCs con- 
sist of a consumer majority, mostly from health 
system agencies’ representatives who are ap- 
pointed by the Governor, and are responsible for 
approving the State health plan which is submitted 
by the SHPDA. 

State health planning and development agency 
(SHPDA): State-level health planning organiza- 
tions which are federally mandated to develop and 
implement statewide health plans in consonance 
with national goals and local planning objectives. 
SHPDAs must approve or disapprove health sys- 
tem agencies’ recommendations regarding certifi- 
cate of need for large new capital investments. 

Willingness-to-pay: An economic concept used to as- 
sess the monetary value of life in terms of what an 
individual is “willing to pay” to prolong life or 
postpone death. The willingness-to-pay technique 
is used to try to assess how much an individual val- 
ues his or her own life. Sometimes, the technique is 
also used to assess how much an individual is will- 
ing to pay to decrease risk to others or to prolong 
others’ lives. 
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Office of Technology Assessment 


The Office of Technology Assessment (OTA) was created in 1972 
as an advisory arm of Congress. OTA’s basic function is to help legislative 
policymakers anticipate and plan for the consequences of technological 
changes and to examine the many ways, expected and unexpected, in 
which technology affects people’s lives. The assessment of technology 
calls for exploration of the physical, biological, economic, social, and po- 
litical impacts which can result from applications of scientific knowledge. 
OTA provides Congress with independent and timely information about 
the potential effects—both beneficial and harmful—of technological ap- 
plications. 


Requests for studies are made by chairmen of standing committees 
of the House of Representatives or Senate; by the Technology Assessment 
Board, the governing body of OTA; or by the Director of OTA in consul- 
tation with the Board. 


The Technology Assessment Board is composed of six members of 
the House, six members of the Senate, and the OTA Director, who is a 
nonvoting member. 


OTA currently has underway studies in nine general areas— ener- 
gy; materials; international security and commerce; food and renewable 
resources; genetics and population; health; space, telecommunication, 
and information systems; oceans; and transportation. 
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